Help:Cloud Services introduction
Wikimedia Cloud Services (WMCS) provides tools, services, and support for technical collaborators who want to contribute to Wikimedia software projects. Use Cloud Services to host your software tools for the Wikimedia movement, without charge.
What can you do with Cloud Services?
Host tools on Wikimedia servers
Tools and bots make it easier to edit and maintain Wikimedia projects. For developers who support Wikimedia projects by developing tools and bots, Toolforge provides the following features:
- Free, reliable, and scalable shared hosting, including web servers, databases and other data storage
- A distributed job processing system
- Support for multiple users to collaboratively maintain and manage tools
To use Toolforge you need:
- Some programming knowledge
- An understanding of Unix command line
To get started, visit Help:Toolforge. Or, learn more about creating bots.
Run scripts and visualize data
PAWS is a Jupyter notebook installation hosted by Wikimedia. PAWS notebooks can be used for creating tutorials, running live code, creating data visualizations, running basic bots, and more.
A single PAWS notebook is maintained by a single user, but they can be downloaded and forked by other users. To use PAWS you need only a Wikimedia login and a web browser. Knowledge of Python is helpful, but not required.
Administer servers for software development
Open source software projects help the Wikimedia movement by improving core infrastructure (like MediaWiki), powering research and analytics, and supporting Wikimedia operations and software development. For advanced projects that aren't viable in the Toolforge environment, Cloud VPS (Virtual Private Server) provides the following features:
- Free cloud computing environment, powered by OpenStack
- Collaboratively-owned collections of virtual private servers, storage, firewall, and HTTPS proxy resources to projects
- Access to a variety of data services
- Freedom to install packages not provided by Debian or the Wikimedia Foundation
To use Cloud VPS, you need:
- An open source project that isn't viable in the Toolforge environment, or can't be accomplished using other WMCS offerings
- One or more active project maintainers who meet basic requirements
- Advanced programming knowledge
- Advanced experience with Unix command line
- The ability to administer your own servers and manage your project's applications, data, runtime, middleware, and operating systems
To get started, visit Help:Cloud VPS.
What is the difference between Cloud VPS and Toolforge?
Cloud VPS is an Infrastructure as a service (IaaS) solution. It provides virtual machines, storage, firewall, and HTTPS proxy resources to projects. The members of each individual project are responsible for managing applications, data, runtime, middleware, and operating systems themselves. Cloud VPS projects use the domain wmcloud.org (some older projects use the legacy domain wmflabs.org).
Toolforge is a Platform as a service (PaaS) solution. It provides web servers, databases, and a distributed job processing system as managed services for tool maintainers. Toolforge tools use the domain toolforge.org.
Access databases and data dumps
Access wiki databases for tool development
Tools and software hosted on Toolforge and Cloud VPS can directly access public data dumps and production wiki replicas.
- Learn about accessing wiki replica databases from a tool account.
- Learn about accessing dumps through shared storage services for Cloud VPS and Toolforge.
Query wiki replicas and dumps in a browser
- PAWS provides a Jupyter notebook environment you can use to query wiki replicas and dumps, create interactive graphs, and use APIs for analysis.
- Superset and Quarry are web interfaces for querying live replica SQL databases of public Wikimedia wikis.
To use Superset or Quarry you need only a Wikimedia login and a web browser, but you should have a basic understanding of SQL. Learn about SQL queries.
Which service is right for you?
| Activity / Needs | PAWS | Superset | Toolforge | Cloud VPS |
|---|---|---|---|---|
| Data as a service | Data as a service | Platform as a service | Infrastructure as a service | |
| Write scripts and visualize data | ✔ | |||
| Write queries against wiki replica databases | ✔ | ✔ | ✔ | ✔ via Toolforge |
| Access on wiki database dump files | ✔ | ✔ | ||
| Write and run bots | ✔ | ✔ | ✔ if not viable on Toolforge | |
| Run web services | ✔ | ✔ if not viable on Toolforge | ||
| Build tools to improve Wikimedia projects | ✔ | ✔ if not viable on Toolforge | ||
| Schedule or run continuous jobs | ✔ | ✔ if not viable on Toolforge | ||
| Need your own subdomain | ✔ | ✔ | ||
| Work with co-maintainers and co-admins | ✔ | ✔ | ||
| Install packages not provided by Debian or the Wikimedia Foundation | ✔ | |||
| Administer your own virtual server | ✔ | |||
| Platform / Environment | web browser | web browser | terminal | terminal |
| User knowledge | curious—advanced | curious—advanced | intermediate—advanced | advanced |
Before you start
To use Cloud Services products, you must first create a Wikimedia account and a developer account.
Learn more
- 🎬 Video: Wikimedia Cloud Services introduction (2019)
- 📣 Slides: An introduction to Cloud Services presentation (2019)
Communication and support
Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:
- Chat in real time in the IRC channel #wikimedia-cloud connect or the bridged Telegram group
- Discuss via email after you have subscribed to the cloud@ mailing list
- Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
- Read the News wiki page
Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself
Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)