Search servers relocation - 12 august 2010

From Wikitech
Jump to navigation Jump to search

Batch one: searchidx1, search1 - search12

Batch two: search13 - search20

Before moving servers

  • In /home/wikipedia/common/php/wmf-config/lucene.php comment out:
 $wgEnableLucenePrefixSearch = true;

This will re-route ajax suggestions from search boxes back to the database.

Moving batch one

Before turning off and relocating search1-12 do the following:

  • on lvs3, searchpool1 should be "search14, search18, search19, search20", searchpool2 should be "search16", searchpool3 should be "search17". Make sure to backup the old configuration.
  • In /home/wikipedia/common/php/wmf-config/lucene.php replace
  '10.0.3.11' # search 11

with

  '10.0.3.13' # search 13


  • verify everything is ok, then searchidx1 and search1-12 are ready to move

Moving batch two

  • bring up search1-12 and make sure lsearchd is started ("sudo /etc/init.d/lsearchd start")
  • bring up searchidx1 - lsearchd should *not* be running here - I'll start it later
  • revert the changes from previous section, i.e. revert changes on lvs3 and on /home/wikipedia/common/php/wmf-config/lucene.php. This will direct queries back to search1-12.
  • Try some test queries on en.wp, de.wp and commons. Allow up to 15 minutes for search to start working (it might take some time for linux to cache the appropriate pages, will see lots of I/O in ganglia stats). If it still doesn't work after 15 minutes, and no CPU is used on any of search1-12, revert the changes and wait for tomorrow to do the second batch.
  • if everything is fine, search13-20 are ready for relocation and can be switched off

Switching back to original state

  • Turn on search13-20
  • Robert will restore original lsearchd configuration
  • uncomment $wgEnableLucenePrefixSearch = true; (revert #Before moving servers)
  • verify everything works