Search Console Data

What?

It is often important for site owners to understand how their sites perform on search engines. Questions such as "how often was my site shown in search results?", "at which position?", "how often was it clicked on?" are very important for site owners. Recognizing this, several search engine platforms (Google, Bing, and Yandex among others) provide site owners with insights on how their sites perform.

These insights are provided through tools hosted by the search engine companies. Google's product is called "Google Search Console". Bing provides something called "Bing Webmaster Tools", and Yandex has something called "Yandex Webmaster". With small differences, these are largely similar in functionality.

Data provided by these consoles is largely complementary to pageview data that WMF collects. It aids in the understanding of trends in search, and in correlating pageview data with search patterns.

How?

WMF maintains access to search consoles offerings from various search engines and provides access to them as needed. As of this writing, search console data is not imported into the data lake. The only way to access this data is to obtain access to the console user interfaces directly.

Request process

Please note that in order to be granted access to WMF data on search consoles, you must either work for Wikimedia or be a volunteer/contractor/hired to do the work. You will not be granted access to all of the (hundreds of) Wikimedia-managed domains but only a subset thereof that you have a business case for.

Please apply for access as follows:

File a task in Phabricator under the #search-console-access-request project.
User must have a valid NDA on file with WMF legal. Not a Phabricator NDA electronically signed, but a full NDA with our legal department.
- WMF staff automatically sign this as part of their hiring process, all non-staff must have the NDA confirmed before access.
User must have a valid reason for accessing the Google Search Console or Bing Webmaster Tools.
- Please document what you'll be doing, and why.
User should have an end date in mind, or setup an expiry time to check access, typically within a year of access (or less.)
All user requests for non staff must have the sponsorship of a WMF staff person.

Notes for Console administrators

Google Search Console

Please note the admin password for Google Search Console is in the pwstore.
There is no scripted check for expiry at this time. Add user to the maint-announce calendar for their expiry until we have a better system.

Adding Users to Domains

This is a tedious process. Once you login to the Google Search Console you will see a list of every single sub-domain for every single language and project in both HTTP and HTTPS.
Login from an incognito browser window or a profile where you're not already logged in with any other Google account to google search console using the admin account in pwstore. This requires a verification code that will be sent via email.
Select the domain you wish to manage from the search/drop down in the top left of the page
Select settings from the menu, then User and Permissions
In almost all cases, the user should only be given Restricted to the Domain.
Click add user, and then enter the user's email address and click add user. Permission should be 'Restricted'.
User has now been added to that project/subdomain. Repeat as needed for other projects/subdomains.
For NON-WMF ADDITIONS: An entry should be added to the maint-announce calendar, set to email out to the root list and the person who approved the access to the search console for the entry to alert them of expiry. Reach out to the SRE Clinic on-duty to request this. Request SCherukuwada@ to remove the user from each domain.

Removing Users from Domains

Login from an incognito browser window or a profile where you're not already logged in with any other Google account to google search console.
Select the domain you wish to manage from the search/drop down in the top left of the page
Click on Manage Domain, Add or Manage Users for the single domain you want to edit. This pulls up a sub-page.
Click the user in question in the list presented on page, this highlights the user.
Click the delete user button on the highlighted line.

Bing Webmaster Tools

Please note that the admin password for Bing Webmaster tools is in pwstore.
There is no scripted check for expiry at this time. Add user to the maint-announce calendar for their expiry until we have a better system.

Adding Users to Domains

Login from an incognito browser window or a profile where you're not already logged in with any other Google account to Bing Webmaster Tools using the admin account in pwstore.
Once you login to Webmaster Tools you will see a menu item called "User Management" on the left sidebar.
Bing has been configured with all the wikipedia.org domains added together. There is no way to grant access on a per-language-domain basis as of this writing.
Click on "User Management" then on the "Add a User" button and give them "Read Only" access.

IMPORTANT NOTE FOR NON-WMF ADDITIONS: As with Google Search console, we need to ensure that the granted permission has an expiration date. An entry should be added to the maint-announce calendar, set to email out to the root list and the person who approved the access to the search console for the entry to alert them of expiry. Then whoever is on clinic duty for that week needs to login and remove the user from each of the domains.

Removing Users from Domains

Login from an incognito browser window or a profile where you're not already logged in with any other Google account to Bing Webmaster Tools.
Click on "User Management".
Click on the triple-dot menu alongside the user you wish to remove and then click on "Remove User".