Jump to content


From Wikitech

There are several roles needed for a functioning Mobile system.


  • RackApp = The main (currently) Merb Ruby application that makes up "WikimediaMobile"'s core code
  • MobileArticle = A custom transformed version of a Wikipedia article in a reusable state

List of Roles

  • MobileHTTP = A server ready to handle public-facing mobile requests
  • MemcachedServer
  • Mysql = A log database server... only used by the LogParse script

Basic Outline of Request Cycle

  • An HTTP request comes in to MobileHTTP
  • The Proxy Server on port 80 passes the request into the/an RackApp process
    • The RackApp checks with MemcacheServer to see if the MobileArticle is cached
    • If it is not found, then the RackApp uses libcurl to scrape a copy from #{lang}.wikipedia.org and transforms it with libxml2 into a cachable MobileArticle. This is then stored in the cache.
    • The MobileArticle is put in a proper layout by the RackApp and is returned to the user
    • The RackApp writes a log entry in JSON format in a file on MobileHTTP
  • The Proxy Server sends back the document

Basic Outline of Log Parsing

  • A LogParse script is run every 5 minutes
  • The script moves the log file to a tmp location (RackApps are still writing to it)
  • The script sends a USR1 signal to the RackApp cluster which causes them to write to a new file
  • The script parses the JSON-formatted log file and merges the information into a single stat entry
  • The stat entry (makes up a row in the db) is written to the Mysql database
  • The temporary log file is deleted