Swift/Server layout

From Wikitech

Swift Server Hardware, Partitioning, and OS information

There are two types of servers involved in a Swift cluster - the front end proxy (ms-fe, standing for media store front end) and the back end storage node (ms-be).

ms-fe

The front end proxies are running on high performance misc servers built with:

  • two 250GB disks
  • 16GB RAM

The disks are partitioned using the raid1-250G-1partition.cfg partman recipe:

  • 200GB partition on each disk software RAID 1 for /, ext3
  • the rest for two swap partitions (one on each disk)

ms-be

The back end storage nodes are running on Dell R720xd machines with:

  • twelve 2TB disks
  • two ssds 150GB each (on some hosts)
  • 48GB RAM

For hosts with ssds, the disks are partitioned such that

  • the last two disks have a 55GB partition for /, set up with software RAID 1, ext3
  • the last two disks have a small (1GB) partition for swap
  • the last two disks have a small (93GB) partition formatted for account and container storage
  • all other disks are formatted with one large XFS partition for swift object storage.

The reason for this is that the ssds show up as /dev/sdm and /dev/sdn instead of /dev/sda and /dev/sdb, because of where they can be cabled up in the box.

Hosts without ssds are partitioned such that

  • the first two disks have a 60GB partition for /, set up with software RAID 1, ext3
  • the first two disks have a small (1G) partition for swap
  • the first two disks have a large (the rest, or ~1.95T) partition formatted for object storage
  • all other disks are formatted with one large XFS partition for swift object storage.

The partman recipe (ms-be-ssd.cfg or ms-be.cfg) only partitions the first or last two disks and doesn't create the XFS partition. Puppet partitions the rest.



The below applies only to previously installed hosts; these will soon be replaced.

The back end storage nodes are running on Dell C2100 machines with:

  • twelve 2TB disks
  • 48GB RAM

The disks are partitioned such that

  • the first two disks have a small (1MB) partition for grub
  • the first two disks have a 250GB partition for /, set up with software RAID 1, ext3
  • the first two disks have a small (4G) partition for swap
  • the first two disks have a large (the rest, or ~1.75T) partition formatted for object storage
  • all other disks are formatted with one large XFS partition for swift object storage.

The partman recipe (raid1-2TB-1partition.cfg) only partitions the first two disks and doesn't create the XFS partition. Puppet partitions the rest.