Jump to content

Swift/Preliminary test plan

From Wikitech
The "Swift" project is Current as of 2012-04-01. Owner: Bhartshorne. See also RT:1384

This page documents a preliminary test plan to get a basic handle on how swift performs, as well as a battery of functional tests to verify swift is working correctly.

Background information

Some statistics from ms5, to be used as a starting point for the tests

  • How many images per second does ms5 serve? about 110/s at peak (from ganglia
  • How many requests per second does ms5 send back to the image scalers? about 50/s at peak (from ganglia
  • How much storage does ms5 have? 10T, 8T used. (ms7: 27T, 18T used)
  • What is the rate of disk consumption on ms5? Historic: 0.3T/mo. Current: 1.2T/mo (ms7: 1T/mo)
  • How long does it take to request 1000 images on ms5 (sequentially)?
    • note - this test is performed during normal load, so it's actually 1000 images plus regular load
  • How long does it take to request 1000 images on ms5 (with 20 clients)?
    • note - this test is performed during normal load, so it's actually 1000 images plus regular load

Test platforms

First iteration

three machines using high performance misc servers

  • X CPU
  • Y RAM
  • 2 disks

two machines are configured as proxy machines, three as storage nodes (the two proxy servers are also storage nodes)

Second iteration (proposed)

separate proxy nodes from storage nodes - five high performance misc servers (two proxy, three storage)

production config (proposed)

  • three proxy nodes - high performance misc
  • 6 storage nodes - ES hardware (grow to 10 over 1yr)

Tests

speed

a list of 30,000 images (pulled from a 3min tcpdump off ms5) is at fenari:~ben/swift/list-of-images. This can serve as input to performance tests.

(30,000 chosen because peak load on ms5 is ~100qps == 6000qpm. 5 minutes =~ 30k.)

  • How many images per second can swift serve?
  • How long does it take to request 30000 images on swift (sequentially)?
  • How long does it take to request 30000 images on swift (with 20 clients)?
  • How does swift maintenance (rebalancing, etc) affect performance? (fail out a disk and retest 1000 image retrieval)

functional against swift directly

  • upload file
  • retrieve file
  • rename file
  • delete file

functional through mediawiki - thumbnails

  • request a thumb that exists on swift
  • request a thumb that doesn't exist on swift (but whose original does exist), watch it get created and stored
    • re-request, verify it comes from local disk
  • request a thumb for an original image that doesn't exist, verify correct error handling
  • purge the image cache, verify the image gets deleted from swift

functional through mediawiki - originals

  • upload file
  • upload new version of file
  • revert to previous version of file
  • rename file
  • delete file