Obsolete:Db20 to NFS1-2 migration plan

From Wikitech
This page contains historical information. It may be outdated or unreliable.

NFS home DRBD migration plan.

Hosts

  • db20 - old server
  • nfs1 - fresh host
  • nfs2 - host replicating from db20

Differences

  • Move replication from db20->nfs2 to nfs2->nfs1
  • Use protocol C, not A

Plan

Preparation

  • Check DRAC access
  • Prepare nfs1 for drbd
  • Add DNS entries for new internal IPs
  • Set up cross link networking between nfs1/nfs2
    • Protocol C, not A
    • replication rate 110M
  • Write final configuration files for nfs1/nfs2
  • Install /etc/exports on nfs1/nfs2
  • Install NFS kernel server on nfs1/nfs2

IP plan

; Cross link between nfs1/nfs2: 10.3.0.0/30

1       1H      IN PTR  nfs1-xlink0.pmtpa.wmnet.
2       1H      IN PTR  nfs2-xlink0.pmtpa.wmnet.

nfs1-xlink0     1H      IN A            10.3.0.1
nfs2-xlink0     1H      IN A            10.3.0.2

New drbd.conf

# drbd.conf for replication between nfs1 and nfs2
# Written on 2009/11/23 by Mark Bergsma <mark@wikimedia.org>

global {
	usage-count no;
}

common {
	protocol C;
}

resource nfshome {
	on nfs1 {
		device /dev/drbd1;
		disk /dev/mapper/mirror-nfshome;
		address 10.3.0.1:7789;
		meta-disk internal;
	}

	on nfs2 {
		device /dev/drbd1;
		disk /dev/mapper/mirror-nfshome;
		address 10.3.0.2:7789;
		meta-disk internal;
	}
}

Migration

On nfs1:

  • Install new /etc/drbd.conf
  • Create device meta-data:
drbdadm create-md nfshome
  • Attach and setup new resource, but do not connect yet:
drbdadm attach nfshome
drbdadm syncer nfshome

On db20:

  • Stop NFS kernel server
  • Remove NFS service IP 10.0.5.8:
ip addr del 10.0.5.8/32 dev eth0
  • Stop userland processes using /home (fuser -m)
  • Unmount /home
  • Comment /home mount in /etc/fstab
  • Check consistency of nfshome replication (cat /proc/drbdadm)
  • Disable resource:
drbdadm down nfshome
  • Save the meta-data in a text file:
drbdadm dump-md nfshome > /root/db20-nfs2-drbd.metadata

nfs2 should now have an exact block level replica of the /home filesystem

On nfs2:

  • Check consistency of nfshome replication (cat /proc/drbdadm)
  • Disable resource:
drbdadm down nfshome
  • Save the meta-data in a text file:
drbdadm dump-md nfshome > /root/db20-nfs2-drbd.metadata
  • Install new /etc/drbd.conf
  • Create device meta-data:
drbdadm create-md nfshome
  • Enable the new resource:
drbdadm attach nfshome
drbdadm syncer nfshome
drbdadm connect nfshome

On nfs1:

  • Connect the new resource:
drbdadm connect nfshome

On nfs2:

  • Start initial device synchronisation (ON NFS2 ONLY!):
drbdadm -- --overwrite-data-of-peer primary nfshome
  • Check replication status (cat /proc/drbd)
  • Add /etc/fstab entry:
/dev/drbd1	/home           jfs     defaults        0       2
  • Mount /home:
mount /home
  • Migrate NFS service IP 10.0.5.8 to nfs2:
ip addr add 10.0.5.8/32 dev eth0
  • Configure service IP on nfs2 on bootup. Add entry to /etc/networking/interfaces eth0:
up ip addr add 10.0.5.8/32 dev $IFACE
  • Start NFS kernel server

Wrap up

  • Migrate udp2log, /usr/local/bin/demux.py from db20 to nfs1 and nfs2
  • Write documentation on wikitech about the DRBD setup

Rollback

If anything goes wrong, the filesystem on /dev/drbd1, or alternatively /dev/sda6 should be in tact and can be mounted. NFS service can then be restored quickly.

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Obsolete:Db20_to_NFS1-2_migration_plan&oldid=132532"