API Portal

From Wikitech

The API Portal is an API documentation and OAuth credential management wiki run by the Wikimedia Foundation.

Terminology

Wikimedia API
The collection of API endpoints served through api.wikimedia.org is referred to as “the Wikimedia API”. While this does create confusion with the MediaWiki Action API, MediaWiki REST API, and the Wikimedia RESTBase API, keeping the name simple helps put aside that complexity and reinforces the preferential status of endpoints served through api.wikimedia.org.
Namespace
An element of the API URL that indicates a collection of endpoints that share a version number. Endpoints within a namespace may not have a common origin in the system. For example, the page description endpoints and the MediaWiki Core REST API endpoints are both nested under the "core" namespace although they live in different services. This means that if any endpoint in the core namespace undergoes a backwards-incompatible change, all endpoints in the namespace must be made available under a new version number.
API key
A client registered through the OAuth extension on Meta Wiki, including both owner-only and non-owner-only clients

Permissions

All logged-in users
Read pages
Edit talk pages
Documentation editors[1]
Edit all pages

As of January 2021, the documentation editors group is open to anyone who requests to join.

Routing

Like the API Gateway, the API Portal is available through api.wikimedia.org. This is handled by Envoy, which routes any requests to api.wikimedia.org/wiki/... to the API Portal. This means that some special routes must be manually added to Envoy in order to work, for example: static/favicon/wmf.ico.

Adding an API

APIs added to the API Gateway should be documented in the API Portal. To add an API to the Portal, open a Phabricator task tagged with #API-Portal. In the task:

  • List the endpoints for the new API as they will be served through the gateway (with namespace and version)
  • Link to any existing documentation for the API
  • Link to the API source
  • Include any applicable deadlines

In most cases, an API Portal contributor (likely APaskulin (WMF)) will be available to help add the API to the Portal. If no one is available or if you'd like to add the docs yourself, use the self-service process.

Self-service process

To edit the API Portal, you'll need to join the documentation editors group. To do this, post a request to the Contributing discussion page.

This process has been temporarily deprecated while the API Portal is in an interim state pending product management changes. Instead, please follow the #Adding_an_API process.

API Gateway endpoints are organized by namespace. For example, the MediaWiki REST API endpoints are mounted in the core namespace (https://api.wikimedia.org/core/v1/wikipedia/en/page/Earth) while the Wikifeeds endpoints are under the feed namespace (https://api.wikimedia.org/feed/v1/wikipedia/en/featured/2021/01/11). Each namespace has a landing page in the reference docs with docs for individual objects and endpoints nested below. To break up long lists of endpoints, namespaces can be organized into endpoint groups with their own landing pages. For example, the core namespace includes several endpoint groups, including the page endpoints and search endpoints. Endpoint groups should be named by the URL elements that identify them.

Within an endpoint group, each endpoint is documented on a separate page. if an object schema is shared between endpoints, it can be documented on its own page and cross-referenced from the endpoints that use it. If an object schema is used by a single endpoint, document it within the endpoint page. For example, the core namespace's page object is documented on a dedicated page because it is returned by several endpoints, while the page history diff object is documented only on the compare revisions endpoint page.

At the same level as an endpoint group landing page, you can add technical topic pages. Because API Gateway endpoints are proxied from various services, differences in behavior and features may occur. These topic pages can help present this information as specific to that namespace.

Here's a skeleton of how the docs for a namespace are structured:

  • Namespace landing page
    • Namespace-specific technical topic
    • Endpoint group landing page
      • Object page
      • Endpoint page
      • Endpoint page

Reference doc page titles start with API reference/{Namespace}/{Endpoint group}, including the endpoint group page in the structure if applicable. To create individual pages, copy a similar page in the reference docs, and modify it. You'll need to change the DEFAULTSORT parameter at the bottom of each page to a unique position in the sidebar. If two pages share a DEFAULTSORT parameter value at the same level, they will not appear in the sidebar correctly.

Finally, once you've added the documentation pages, update the changelog to announce the new endpoints. Any change to endpoint behavior should be posted to the changelog.

Architecture

Wikimedia API Gateway architecture diagram

Components

MediaWiki
The API Portal is MediaWiki instance at api.wikimedia.org (beta: api.wikimedia.beta.wmflabs.org)
WikimediaApiPortal skin
A MediaWiki skin providing the look-and-feel of the API Portal
WikimediaApiPortalOAuth extension
A MediaWiki extension running on the API Portal that interfaces with the OAuth extension on Meta-Wiki
OAuth extension on Meta-Wiki
Responsible for storing and managing OAuth clients

Content strategy

The content strategy for the API Portal is in an interim state pending product management changes. See phab:T289791 for more information.

Duplication

Because the API Gateway is a proxy, documentation in the API Portal is often partially duplicated with documentation stored elsewhere. For example, the Core REST API endpoints are documented on mediawiki.org and on the API Portal. There are small differences between these two sets of documentation, including URLs and information about behaviors that can be configured at the MediaWiki level.

User agents in examples

The user agent policy requires that user agents be included in API requests for analytics purposes. In the Portal, all non-curl example should include user agents with the exception of JavaScript examples (phab:T268791).

API key management and OAuth integration

The API Portal includes a special page, Special:AppManagement, to manage Wikimedia OAuth client credentials. The canonical source of truth for OAuth clients is the OAuth extension on Meta. The WikimediaApiPortalOAuth extension integrates the API Portal with Meta through the OAuth extension REST endpoints.

The API Portal offers fewer options for creating clients than the original form on Meta. Here is the spec detailing which fields are used by the API Portal when creating clients:

  • name: Input on the form
  • version: Always 2.0
  • description: Input on the form
  • wiki: Always * for all wikis
  • owner only: Set if the "personal API key" option is selected
  • callback URI: Input on the form
  • callback prefix: Set to false (This was set to true prior to 886848.)
  • email: Automatically set to the user's email
  • confidential: Set to false if the "client-side app" option is selected
  • grant types: If the "personal API key" option is selected, set to client credentials. If the other options are selected, set to authorization code, refresh token, and client credentials.
  • scopes:[2]
    • Default: basic
    • If "Create and edit pages" is selected: createeditmovepage
    • If "Edit protected pages" is selected: editprotected

Configuration

Navigation

To change the links in the navigation bar at the top of the page, edit MediaWiki:Sidebar.

Design

The design of the API Portal is based on the Wikimedia design style guide, the design of design.wikimedia.org, and OOUI. The original design was created using Axure, a graphical prototyping tool, and combined with OOUI styles to produce the final look and feel.

The API Portal is designed to be used with the WikimediaApiPortal skin but should still be basically functional with other skins. The API Portal has other skins disabled in basic user preferences, so only users with global skin preferences will experience the site with other skins.

In addition to the styles included in the skin and in OOUI, Common.CSS includes styles that help the Portal display correctly. (See phab:T266279.)

Content width

To improve readability, content on non-reference pages should be limited to a max width of 775px.

Workflow

Work related to the API Portal is managed through the API Portal board in Phabricator. In general, tasks on the API Portal board should be self contained; we avoid using the board to manage epics or initiatives. Any unassigned tasks are free for anyone to pick up and start working on. The on-wiki contributing process lives on Community/Contributing.

Skin and extension tasks

Tasks related to the API Portal skin are tagged with #MediaWiki-skins-WikimediaApiPortal. Tasks related to the API Portal extension are tagged with #MediaWiki-extensions-WikimediaApiPortalOAuth. Managing these components through the central API Portal workboard gives us a complete picture of the work for the whole project and avoids having to maintain multiple boards.

Columns

  • Backlog - Default column where tasks appear before they are triaged
  • Tech debt - Bugs, refactoring, code quality improvements, and other technical tasks
  • New features - New functionality to add to the site, including improvements to existing features
  • Content - Tasks related to the documentation and other content on the API Portal wiki
  • Discuss - Investigations, planning, and discussions related to the API Portal

Priorities for bugs and technical issues

  • Low - Issues that don’t impact site functionality, including cosmetic changes
  • Medium - Issues that impact general site functionality, including impacts to accessibility and user experience
  • High - Issues that impact core site functionality (ability to read docs and create and manage API keys)

Local development

To run the API Portal locally, you'll need these components installed:

  • MediaWiki
  • MediaWiki OAuth extension
  • Mediawiki WikimediaApiPortalOAuth extension
  • MediaWiki WikimediaApiPortal skin

Workarounds

Email confirmation

If you don't have email confirmation working for your MediaWiki instance, you won't be able to create OAuth clients locally. In a pinch, you can avoid this requirement by making these temporary code changes.

In mediawiki/extensions/OAuth/src/Frontend/SpecialPages/SpecialMWOAuthConsumerRegistration.php
Change:
	public function userCanExecute( User $user ) {
		return $user->isEmailConfirmed();
	}
To:
	public function userCanExecute( User $user ) {
		return true;
	}
In mediawiki/extensions/OAuth/src/Control/ConsumerSubmitControl.php
Change:
			if ( !$permissionManager->userHasRight( $user, 'mwoauthproposeconsumer' ) ) {
				return $this->failure( 'permission_denied', 'badaccess-group0' );
			} elseif ( !$user->isEmailConfirmed() ) {
To: 
			if ( !$permissionManager->userHasRight( $user, 'mwoauthproposeconsumer' ) ) {
				return $this->failure( 'permission_denied', 'badaccess-group0' );
			} elseif ( $user->isEmailConfirmed() ) {

(remove exclamation mark)

In mediawiki/extensions/WikimediaApiPortalOAuth/src/Special/AppManagement.php
Comment out:
		/**if ( !$this->getUser()->isEmailConfirmed() ) {
			$output->addHTML(
				$this->msg( 'wikimediaapiportaloauth-email-not-confirmed' )
			);
			return;
		}**/

Background

Original scope (March 2020)

The API Portal was originally created by the Wikimedia Foundation Platform Engineering Team in 2020 as part of the Platform Evolution program.[3] It had these objectives:

  • Provide documentation for API endpoints served through the API Gateway and allow for creating and managing Wikimedia OAuth 2.0 clients through a single site.
  • Use a visual design that resembles popular API documentation sites, based on the Wikimedia design style guide.
  • Minimize cognitive overhead by reducing reliance on information stored in other wikis.

Original content strategy (March 2020)

Organization

  • Home page
    • Navigation landing page
      • Section landing page
        • Documentation page
          • Documentation sub-page

Outline

  • Main page
  • Use cases landing page
    • Daily featured content
    • Current events
    • Most read articles
    • Education
    • Travel
  • Documentation landing page
    • Getting started
      • Wikimedia projects
      • Reusing free content
      • Authentication
      • Managing API keys
      • Rate limits
    • Code samples
      • Featured content
      • Search Wikipedia
      • Images and media
    • Best practices
      • Stability policy
      • Security
      • Conditional requests
      • Tools and bots
  • API reference landing page
    • Changelog
    • [Namespace landing pages and nested reference pages]
  • Community
    • Contributing
    • About
    • FAQ

Content types

Main page
Use case pages
These are landing pages that provide alternate paths into the documentation based on keywords that groups of users are likely to identify with. This can include industries (like travel and education), capabilities (like featured content), or technical areas (like machine learning).
Documentation pages
Code samples
Landing pages
References pages
Either endpoint or object pages in the API reference

Acceptance testing (November 2020)

Acceptance test cases for WikimediaApiPortalOAuth and updates to OAuth extension

task T261702

  • User with an unconfirmed email address is prompted to confirm their email before they can create a client
  • User with a confirmed email address can create a client
  • When creating an owner-only client, the form requires a client name, description, and terms agreement.
  • After creating an owner-only client, a dialog appears with the client ID, secret, and token.
  • When creating a non-owner-only client, the form requires a client name, description, return URL, confidential status, and terms agreement.
  • After creating a non-owner-only client, a dialog appears with the client ID and secret.
  • Existing OAuth 2.0 clients appear in the client list with their name, created date, and status
  • Existing OAuth 1.0 clients do not appear in the client list
  • The client details view shows the client ID
  • For an owner-only client, the client secret and token can be reset using the reset option
  • For a non-owner-only client, the client secret can be reset using the reset option
  • Changes in client status are reflected in the Portal

MVP validation (February 2021)

End-to-end test cases for MVP validation

task T268257

Using OAuth clients created via Meta
  • Owner-only clients (access token only)
    • Read
    • Write
  • OAuth 2.0 client credentials workflow (two-step authentication)
    • Read (Works only after admin approval)
    • Write
  • OAuth 2.0 authorization code workflow (three-step authentication)
    • Read (Works with only the creators account prior to approval)
    • Write
    • Refresh
Using OAuth clients created via API Portal
  • Owner-only clients (access token only)
    • Read
    • Write
  • OAuth 2.0 client credentials workflow (two-step authentication)
    • Read
    • Write
  • OAuth 2.0 authorization code workflow (three-step authentication)
    • Read
    • Write
    • Refresh

Scope v2 (August 2021)

See task T289183.

Mission statement

As part of the API Platform, the API Portal is an information hub for API producers and consumers to discover, consume, and manage Wikimedia APIs.

Personas

  • API producers
  • API consumers
  • Technical writers
  • Product managers

Capabilities

  • Allow for the discovery of ALL (new, old, experimental, public, internal, Gateway, non-Gateway, WMF, non-WMF) Wikimedia HTTP APIs through an API catalog
  • Serve API reference documentation created using the to-be-developed API Platform documentation tooling
  • Act as a multilingual wiki for API-related learning materials, like tutorials and best practices
  • Create and manage Wikimedia OAuth 2.0 clients

Maintainers

References

  1. ↑ https://api.wikimedia.org/w/index.php?title=Special:ListUsers&group=docseditor
  2. ↑ https://github.com/wikimedia/mediawiki-extensions-WikimediaApiPortalOAuth/blob/master/resources/widget/booklet/page/NewClient.js#L238-L250
  3. ↑ mw:Core_Platform_Team/Initiatives/API_Gateway/Documentation_Plan
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=API_Portal&oldid=2086315"