Zedler/Replication
The replication is done with 2 different programs. At first with the normal mysql-replication-program and at secound with a tool programmed by river (trainwreck). The reason for this is, that mysql can only replicated one master at the same time.
So the mysql-replication is used to replicate the s2-cluser (dewiki and a few others) on zedler, and s1 (enwiki) on yarrow. The replication tool replicates s3 (frwiki and the rest) on yarrow.
The connection to the masters in Florida is done by a few ssh-tunnels. These ssh-tunnels are started by a little-bash-script /usr/local/sbin/sqltunnel". The tunnels listen at different ports at zedler and forward the traffic to different masters in Florida. 330X is the port-schema at zedler. X stand for the cluster-number (s1, s2 and s3). See the table:
| Cluster | Wikis | Port at zedler |
Replication- program |
|---|---|---|---|
| s1 | enwiki | 3301 | MySQL |
| s2 | dewiki *1 | 3302 | MySQL |
| s3 | frwiki *2 | 3303 | trainrweck |
*1: and a few other wikis
*2: and the rest
Password
The mysql-user-user and -password, that are used for replication, can find at "/root/repl". Never give it to anyone!
MySQL-Replication
The MySQL-Replication is done normaly by mysql. Mysql listen to localhost and port 3302. Replication can be start with "slave start;" and stop with "slave stop;" at the mysql-bash. For the logfile-position see Zedler/MySQL#Logfile. More informations about mysql-replication can find at [1].
DaB.s Replication-Program
DaBs programm can be find at "/root/syncdb/". Normaly, each instance runs in a screen and each instance can be start with "startit.s1" for s1 or "startit.s3" for s3. That starts a little bash-script that runs the java-programm in a loop. The logfiles can be find at "/root/syncdb/logs/"; the config-files can find at "/root/syncdb/config/". The programm is work-in-progress.
Howto to restart replication after the master-db-server changed
- Ask the wikimedia-server-admins for the cluster, the new mastername, the new logfile-name and the new logfileposition.
Stopping
- Login to zedler
- Login to mysql.
- (run "
/bin/bash" if you like). - Run "
slave stop;" to stop mysql-replication - Leave mysql with "
quit;" (difficult, isn't it? ;)) - Run "
ps -ef|grep startit.s" for searching for all dabs-replication-program-loops - Kill all dabs-replication-program-loops with "
kill -9 processid processid processid....". - Run "
ps -ef|grep syncdb" for searching for all dabs-replication-programs - Kill all dabs-replication-programs with "
kill -9 processid processid processid....". - Run "
ps -ef|grep sqltunnel" for searching the sql-tunnel-script. - Kill all sql-tunnel-processes with "
kill -9 processid processid processid....". - Run "
ps -ef|grep ssh|grep 3306" for searching the ssh-tunnels. - Kill all sql-tunnels with "
kill -9 processid processid processid....".
Changing
- Open "
/usr/local/sbin/sqltunnel" with your favorite editor. - Change the startcommands. For example: Change "
start_tunnel X 3301" to "start_tunnel Y 3301" for a masterchange from X to Y. - Save the file.
Starting
- Run the sql-tunnel-script with "
nohup /usr/local/sbin/sqltunnel&". - Login to mysql again.
- If s2 was changed: Change the logfile-name and the logfileposition for mysql. For example, if the name changed to Y-bin.001 and the position to 4, then run "
CHANGE MASTER TO MASTER_LOG_FILE='Y-bin.001', MASTER_LOG_POS=4;". See [2] for more details. - Start mysql-replication with "
slave start;" - If s1 or s3 was changed: Change the db with "
use sync_34176;". Then update the position with "UPDATE pos SET logdatei="logfilename", logpos=logfilepos where pos_id=X". X is 1 or 3 (1 for s1 and s3). Be very sure, what X you use! - Leave mysql with "
quit;" - Open a screen with "
screen" - Run "
/bin/bash" - Goto "
/root/syncdb" - Start s3-replication with "
./startit.s3.sh" - Leave the screen with "
Controll+a and d" (d without controll-pressed) - Repeat the last 5 steps for s1 (replace all s3 with s1)
- Be happy :)