Maps/Infrastructure plans

This page describes plans and not the current state of maps servers

Constraints

When updating vector tile styles you may run into a scenario where there is a schema change, and the new style and old style vector and raster parts are not forward or backwards compatible with each other. These should be avoided, but are sometimes necessary.

Additionally, we need to withstand the following while handling a schema change:

- Two specific servers failing

- One specific server failing and a datacenter failing

Desirable properties are the abilities to

- take a database server offline to run maintenance (e.g. reclustering),

- reimport with updated software

- upgrade other software

Architecture

The maps cluster will consist of four machines in each datacenter. Under normal operation, requests can go to any of the four machines. When a schema upgrade is necessary, the following needs to be done

Recommendations 2022-12

These are the shared recommendations discussed in a meeting between WMF Product Infrastructure developers and WMDE Technical Wishes developers:

• task T216826: Service containerization
  • Moving to k8s will make it easier to upgrade Node (now and in the future).
  • Split geoshapes into a separate service.
    • Containerization made it much easier to refactor the service. There's a proof-of-concept port into TypeScript.
  • Will lower the maintenance barrier.
• task T321789: Node 14+
  • Any earlier Node is already end-of-life.
  • This upgrade will be a strategic requirement for moving to mapslibre or generally, upgrading upstream packages.
• task T210815: Mapnik 3.1 which is the final "LTS" for the old stack.
• task T326407: Plan what a mapslibre.org migration would look like
  • Will be a lot of work if we keep our current schema and styles.
  • Have pitched https://openmaptiles.org/ in the past. Has a very similar style available: https://openmaptiles.org/styles/#osm-bright
  • Feasible. The API does allow us to port what has already been implemented.
  • Need to keep language support in mind.
    • Mini-investigation into benchmarking language support with the WebGL renderer which is part of mapslibre: T281230
• task T263854: Client-side rendering of vector tiles
  • Looked into client-side rendering in the past, the obstacle was a product concern of "anything that will change the experience".
  • The performance tradeoff depends on loading a client library for maps rendering, maybe.
  • Now that we have vector tiles available, it's a short leap to making a client-side rendering proof-of-concept.
• task T274388: Separate geoshapes DB from map rendering DB using imposm capabilities
  • Depends on the geoshapes split above.
  • Will make it much easier to maintain the full-planet OSM sync database for rendering.
  • Geoshape database will be smaller.
• Could look at enabling "nearby" on Android again.
• task T323113: Current work to expand geoshapes at page parse time
  • Is expected to take some load off of the servers.
  • Could result in eliminating components eg. the mapdata library.

Devs agree that the most important next step is containerization. WMDE will have to drop out but we hope this work is continued.