MobileFrontend/Deployment
MobileFrontend rides the regular WMF deployment train. For out-of-band deployments, MobileFrontend has some deployment procedures/considerations beyond the normal WMF deployment procedures. This page serves to document these and is meant to be an addendum to How to deploy code.
Deployment rubric
In general, MobileFrontend rides the regular WMF deployment train. Below is a rubric for making out-of band deployments for bug fixes, etc:
- Deploy ASAP: Patches to fix critical functionality, security issues, or legal issues. Eg mobile edits are not being saved or article content is no longer accessible.
- Deploy in next Lightning Deployment: Patches for semi-critical functionality that is broken but doesn't impede basic usage of critical site components (reading, editing). EG: the watchlist star on an article doesn't accurately indicate if an article is being watched.
- Deploy along with the next regular train: Patches for annoyances (styling not quite right, obscure javascript errors), non-security/legal issues in beta/alpha versions; all other patchsets.
ResourceLoader
There is a bug in resource loader (#37812) that prevents module timestamps from properly updating in certain situations (eg when a resource path gets updated or a resource gets removed and nothing else in the module changes). This can result in outdated JS/CSS getting served. To fix this, first touch one of the resources of the affected module on Fenari:
$ cd /home/wikipedia/common $ touch php-1.21wmf1/extensions/MobileFrontend/javascripts/modules/mf-search.js
Then, sync the file to the cluster:
$ sync-file php-1.21wmf1/extensions/MobileFrontend/javascripts/modules/mf-search.js 'touch file'
You may need to wait up to 5 minutes for the cached resources to expire.
Mobile varnish cache
If HTML in the mobile version of the site is getting updated during a MobileFrontend deployment, you will likely need to get someone to manually purge the mobile varnish cache. Ping someone in ops (binasher, mutante, and LeslieCarr have typically helped me with this) to perform the flush. Instructions can be found on MobileFrontend#Flushing_the_cache.
Performance
On the WMF cluster, it is vital to maintain a reasonable cache hit ratio of the mobile varnish servers. Mobile varnish stats can be monitored using Ganglia. Keep an eye on this for a while after a deployment.
Apache logs
During and after a deployment, you should keep an eye on the apache logs on Fenari. We have had a number of instances of a deployment resulting in spamming the apache logs with errors and warnings that were not caught until some time after deployment. From fenari:
$ fatalmonitor
Or if you prefer to tail the log apache logs:
$ tail -f /home/wikipedia/logs/syslog/apache.log
You can also view a graph of recent exceptions and fatal errors on the cluster on Grafana.