Help:Toolforge/Developing successful tools

From Wikitech
Jump to navigation Jump to search

Overview

This page provides tips to help you develop successful tools on Toolforge. If you have useful advice share here!

FLOSS Best Practices for Bots and Tools poster.pdf

Pick a license

All code in the ‘tools’ project must be published under an OSI approved open source license. Please add a license at the beginning!

A clear license is very important. It explains the the rights the you are willing to grant to others who want to use or modify the software you built. Based on the general principles of the Wikimedia movement.

To learn more about choosing a license for Wikimedia see Wikimedia movement.

The two easiest options for your licenses are:

  1. GPL-3.0+ if you want to ensure that all derivative works are made available under the same license terms (an older GPL license is used for MediaWiki itself) or
  2. MIT if you only want to ensure that your original work is mentioned in a derivative project.

https://choosealicense.com/ is a good explanation of differences between many Free and Open Source licenses. Be aware the some of the licenses described there are not OSI approved however, so make sure to check against the OSI list before using a license for your project.

How to add your license to your source code

License your source code and document that with a LICENSE or COPYING file in the tool's home directory and header comments in the source code.

Publish the code

Sharing files via packages or version control

  • Package shared libraries using the package manager for your implementation language: PHP Composer packages, Python PyPI packages, Ruby gems, etc. This is the recommended and most portable and future-proof method of sharing code with multiple projects.
  • Shared code can be stored in git submodules, which allow users to keep a git repository within another git repository. Sharing code in this way retains the maintainability and other source controls advantages of git. For more information about git submodules, please see the git documentation.

Sharing files via NFS

Programs running on the job grid and in Kubernetes webservice containers have access to shared NFS directories. This shared access can be used to share files between multiple tool accounts. This is the least portable method of sharing and may not be supported in all future Toolforge services.

  • Access to a tool's code can be delegated to other tools by adding the tools as maintainers of the tool that is hosting the code. This can be done at https://toolsadmin.wikimedia.org/tools/id/<tool>/maintainers/. (It may be appropriate to create a new tool to house the shared code.)
  • Shared config or other files may be placed in the /data/project/shared directory, which is readable (and potentially writeable) by all Toolforge tools and users. In this directory are available, for instance:

Have co-maintainers

Find co-maintainers for your tools who can help out at least with starting/stopping jobs when needed.

Write some docs

Document your tool. This is essential for others to know how it functions and to help maintain it into the future.

Create a page in the Tool: namespace documenting the basics of what your tool does and how to start and stop it.

Going beyond

Operating in the open is essential to the success of Toolforge projects.

Planning for success

There are many things to think about when you are planning to build a tool.

Have the accounts you need

Follow the Toolforge quickstart guide to make sure that you have all of the accounts and logins you need to begin developing tools with Toolforge.

Secure passwords and other credentials

Keep passwords and other credentials (OAuth secrets, etc) separated from the main application code so that they are not exposed publicly in your version control system of choice.

Stay small

Make many small tools that each do one specific task rather than a catch-all tool that does many different tasks.

Pick the right development environment

If you will be doing heavy processing (e.g., compiles or tool test runs), please use the development environment (dev.tools.wmflabs.org) instead of the primary login host (login.tools.wmflabs.org) so as to help maintain the interactive performance of the primary login host.

The dev.tools.wmflabs.org host is functionally identical to login.tools.wmflabs.org.

Even when running on dev.tools.wmflabs.org, your processes may be killed without notice by system administrators or automated watch processes if performance of the shared server is severely impacted. Using the job grid is recommended for any heavy processing.

Determine which public version control you will use

You'll want to use public version control for your tool. You can learn more about how to use version control with Toolforge here: Help:Toolforge/Version Control in Toolforge.

Pick a programming language

Do you have experience that might help another user? Please share it (or point to it) here!

Include useful links on a tool's public webpage

If your tool is a web application, you should include some useful links on it for users. For example:

  • Feedback page link,
  • Documentation link,
  • Source code link and license,
  • Authors name(s).

See more ideas and examples in phab:T202949.

Communication and support

We communicate and provide support through several primary channels. Please reach out with questions and to join the conversation.

Communicate with us
Connect Best for
Phabricator Workboard #Cloud-Services Task tracking and bug reporting
IRC Channel #wikimedia-cloud connect General discussion and support
Mailing List cloud@ Information about ongoing initiatives, general discussion and support
Announcement emails cloud-announce@ Information about critical changes (all messages mirrored to cloud@)
News wiki page News Information about major near-term plans
Cloud Services Blog Clouds & Unicorns Learning more details about some of our work
Wikimedia Technology Blog techblog.wikimedia.org News and stories from the Wikimedia technical movement