User:SRodlund/PAWS examples (staging)

Phabricator project: #paws
From Wikitech

This is a staging page for: https://wikitech.wikimedia.org/wiki/PAWS/PAWS_examples_and_recipes

I am currently using this page to compile PAWS resources mainly for Wiki Replicas and API. This is a mix of tutorials, notebooks, and relevant pages.

To Do

Purpose

To help newcomers and current users of PAWS find exisiting notebooks and tutorials that can serve as models for their own work.

Developer/User Stories

  • I am not a developer; I want to perform some basic technical tasks to help improve the wikis I am working on.
  • I am not a developer; I am interested in Wikidata and want to have more skills that allow me to work with it.
  • I am not a developer; I would like to gain some Python programming skills that I can use to contribute to Wikimedia technical projects but will be applicable to other projects outside of Wikimedia
  • I am a developer; and I am looking for additional examples of notebooks that I can use as a basis for my own.
  • I am a researcher and want to access database replicas and work with datasets.

Note: potential audiences in bold

Possible layouts

Thoughts

  • If this is a single page, I would suggest sorting this page by task type and/or audience (label with libray/package ie Pywikibot, numPY, etc).
  • Consider making this a "mini" landing page in the PAWS portal. You could branch out to pages for separate audiences or tasks (though this may be complicating the lightswitch).

Examples to draw from

Staging below


This page is work in progress and will be developed further.

Overview

This page offers a growing number of recipes, how-tos, and example notebooks that you may find useful while learning and exploring PAWS. This page is not meant to be an exhaustive list. There are many examples of public notebooks available in many places. To see all of the notebooks currently hosted on PAWS, check out the public index.

Tracked in Phabricator
Task T266949

There is currently no way to search the public index for specific types of notebooks, but it can be useful to explore them to see what others have done with PAWS.

While the following information is separated by areas of interest, it should be noted that many notebooks utilize a variety of these elements at once.

A notebook may use the Pywikibot library, employ API connections, and utilize wiki replicas.

Key

Color key

A visual key to help keep track of what examples and tutorials are available Example Tutorial API Wikireplicas Datadumps Research & Analysis On-Wiki tasks Pywikibot Wikidata

Areas of Interest

PAWS Tutorials

Following are resources to help you get started with PAWS.

Notebook based tutorials

Wiki replicas (Databases)

Resources for this section

Tutorials and helpers

  • JMos cheatsheet - This has a Database Connections section that is very useful and probably deserves its own tutorial notebook (Note to self -- create a separate notebook just from this section. It's hard to find just based on the title of the cheatsheet, there's no TOC in the doc, and some of the information may be out of date.). Wikireplicas Tutorial Research & Analysis On-Wiki tasks
  • Replica helper - This is a importable notebook that provides simple helpers for performing queries on the labsdb replica databases from PAWS. It is stateful and designed to be easy to use in an interactive setup. Wikireplicas Tutorial Research & Analysis On-Wiki tasks
  • Accessing Database Replicas With Pandas and Sqlalchemy Pandas is a lovely high level library for in-memory data manipulations. In order to get the result of a SQL query as a pandas dataframe use the code provided here.

Example notebooks

Mediawiki APIs

Tutorials

  • Search Wikipedia articles - The MediaWiki REST API lets you build apps and scripts that interact with any MediaWiki-based wiki. In this tutorial, we'll use the REST API search endpoints to search for articles about the Solar System on English Wikipedia. Tutorial API On-Wiki tasks
  • Exploring page history - The MediaWiki REST API lets you build apps and scripts that interact with any MediaWiki-based wiki. In this tutorial, we'll use the REST API page history endpoints to explore the history of articles on English Wikipedia. Tutorial API On-Wiki tasks
  • Wikimedia Feeds API Intro - Many Wikipedias include daily featured articles and other curated content on their homepages. You can see an example of this content on the main page of English, German, and French Wikipedias. The Wikifeeds API lets you access this content programmatically and add high-quality, multilingual content to your apps. Tutorial API On-Wiki tasks

Other ares of interest

Notebooks that use multiple Python libraries

Wikidata

Pywikibot

  • Using Pywikbot with PAWS - A basic introduction to using Pywikibot with PAWS. This tutorial gives you the information you need to get started using a Python 3 notebook or the PAWS terminal. Tutorial Pywikibot
  • An intro to Pywikibot - A notebook based Pywikibot tutorial Tutorial Pywikibot

Example notebooks (This section will likely change)

Notebooks that use datasets

Notebooks that use Pywikibot

Understand users and user behavior on a wiki

Make it easier for editors to organize articles and information

Contribute to Wikidata

Answer interesting questions

  • Teahouse questions -- What kinds of questions do Teahouse users ask? This notebook uses Pywikibot and matplotlib.pyplot to find out. Tutorial Pywikibot Research & Analysis

Further resources and useful pages

Some notes to self

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=User:SRodlund/PAWS_examples_(staging)&oldid=1891650"