User:MSantos/Maps/Tilerator
Tilerator
Tilerator is a backend vector tile pre-generation service that picks up jobs from a Redis job que, copying tiles from a Postgres DB, using sql queries into vector tiles stored in Cassandra. Postgres DBs are set up on each of the maps hosts, one master and 3 slaves. Technically, Tilerator is not even a generator, but rather a "batch copying" service, which takes tiles from one configured source (e.g. a tile generator from SQL), and puts it into another source (e.g. Cassandra tile store).
TileratorUI
TileratorUI is used to inspect maps, including internal data sources, and to add jobs to the Tilerator job queue. Actually, TileratorUI is the same code as Tilerator, but started with a different configuration. Connect to TileratorUI using an ssh tunnel, e.g. ssh -L 6535:localhost:6535 maps-test2001.codfw.wmnet
and navigating to http://localhost:6535. There, you can view any style (use set style to change it), or to schedule a job by setting all relevant fields and Control+Clicking the tile you want to schedule.
- Style is specified in upper left corner
- Set it to
genview
to view tiles generated on the fly. Caution: that means that if you zoom out to low levels tiles can take more that 10 minutes to generate.
- Set it to
- Alt+click (Option+click on Mac) on map to enqueue regeneration jobs.
- This requires
src
anddst
to be set. For the most basic operation, on-demand regeneration of tiles, setsrc
togen
anddst
to whatever Cassandra keyspace is used for tile storage (currentlyv4
). - By default, only tile clicked upon will be regenerated.
- Set
fromZ
andbeforeZ
to regenerate a bunch of layers under the clicked tile.
- This requires
- Click on
source
to view the currently active sources configuration.
See full Tilerator documentation for all commands & parameters.
Generating Tiles
Generate all tiles for zooms 0..7
, using generator gen
, saving into v4
everything including the solid tiles, up to 4 jobs per zoom.
src=gen dst=v4 parts=4 baseZoom=0 fromZoom=0 beforeZoom=8 saveSolid=1
Generated tiles only if they already exist in v2
source, and save them into v4
, on zooms 8..15
, 60 jobs per zoom.
src=gen dst=v4 parts=60 baseZoom=0 fromZoom=8 beforeZoom=16 sourceId=v2
- On existing server, record all existing tiles as a list of tile indexes (path and generatorId need to be adapted)
node /srv/deployment/tilerator/deploy/node_modules/tilerator/scripts/tileshell.js \
--config /etc/tileratorui/config.yaml \
# List all tiles in the "v5" source
-j.generatorId v5 \
# Which zoom to enumerate
-j.zoom 14 \
# File to write indexes to
--dumptiles /home/yurik/all-tiles-14.txt \
# Instead of dumping indexes in "zoom/x/y" format, write one number indexes (0..4^zoom-1)
--dumprawidx \
# If dumptiles file already exists, override it
--dumpoverride
- Instead of generating the entire zoom level, you may want to generate just the tiles in a list (all parameters might need to be adapted)
node /srv/deployment/tilerator/deploy/node_modules/tilerator/scripts/tileshell.js \
--config /etc/tileratorui/config.yaml \
# List of tile indexes, unique and sorted, one per line.
# Indexes can be 0..4^zoom-1
-j.filepath /home/yurik/all-tiles-14.txt \
# All tile indexes in the file belong to zoom 14
# Without this parameter, the file must contain zoom/x/y triplets
-j.fileZoomOverride 14 \
# generate zoom levels 10 <= zoom < 16
-j.fromZoom 10 -j.beforeZoom 16 \
# Copy tiles from "gen" source to "v4" source
-j.generatorId gen -j.storageId v4 \
# If tile already exists in "v4", but "gen" produces an empty tile, delete it in v4
-j.deleteEmpty