SRE/Dc-operations/Common Data center Specifications

From Wikitech

SRE Data Center Operations

DC Operations | About | Projects & Workboards | IRC: #wikimedia-dcops connect
HW Troubleshooting | HW Specific Documentation

This page will outline our preferred specifications for use in data centers.

This is summary data, and is not "live", any actual cable connections, and their cable color and labeling are tracked in Netbox.

Cabling specifications

Color Cable spec Network
Blue Cat6 Production
Green Cat5e Management
Orange Cat5e Serial
Red Cat5e Out of Band Access
Yellow 9/125 Singlemode Fiber All fiber connections
  • Patch Panel Notes:
    • SC Connectors: codfw, eqdfw, eqiad, eqord, eqsin, esams
    • LC Connectors: drmrs
  • Cabling notes:
  • DAC cables are used when possible for in rack connections, otherwise LC-LC fibers with optics.
  • We use pre-made patches in rack (5'/7'/10') for mgmt and production network.
  • We use pre-made patches in rack for serial, and prefer pre-made cables for inter-rack serial connections as well.
    • In the past we made manual cables but those have a higher failure rate. Past scs consoles required special pin outs, but now they just use standard cat5/6 cables.

Cabling options

  • Gear Mounted Top of Rack
  • 12 3M Fiber or 12 10' CAT6 Ethernet Cable
  • 24 2M Fiber or 24 7' CAT6 Ethernet Cable
  • 5 1M Fiber or 5 5' CAT6 Ethernet Cable
  • Gear Mounted Mid-Rack (depreciated, we deployed codfw originally as mid-rack but have found it problematic when rolling upgrades to network infrastructure versus top of rack mounting.
  • 10 1M Fiber or 10 5' CAT6 Ethernet Cable
  • 32 2M Fiber or 32 7' CAT6 Ethernet Cable

Power specifications

All racks are fed with redundant power feeds. Red for the A side and Blue for the B side.

Site Power Feed
codfw 208V 3Phase @ 30A
drmrs 230V 1Phase @ 32A
eqdfw 110V 1Phase @ 30A
eqiad 208V 3Phase @ 30A
eqord 110V 1Phase @ 30A
eqsin 230V 1Phase @ 32A
esams 230V 1Phase @ 32A
ulsfo 208V 1Phase @ 30A

Enclosure specifications

Eqiad racks

  • Depth from front rail to PDU (overall depth for hardware facing the front of the rack): 33"
  • Rittal TS-8 Enclosure: Rittal 9962172
  • Sheet Steel, 800MM W by 2200MM H by 1050MM D. Sand Texture, Black, Perforated front/rear doors, standard rear wall/door, perforated with 4 round cable entry in roof.
  • We have to get three kinds, right side end of row, left side end of row, and enclosures within the row.
  • Inner enclosure: partition wall left side: Right end enclosure: partition left, sidewall right: Left end enclosure: sidewall left
  • Rittal TS - External Baying Connector : Rittal 8800490 : One kit gangs together two enclosures
  • Rittal Roof Mount Cable Manager for 32" Wide TS Enclosure : Rittal 9970016
  • Rittal Button Mount Brackets
  • Rittal 9971067

Codfw racks

  • Depth from front rail to PDU (overall depth for hardware facing the front of the rack): 38"
  • Racksolutions racks purchased via CyrusOne.

All other sites

  • Racks are provided from the vendor in all other datacenter sites.
    • Caching sites are typically 2 adjacent cabinets. Must have full partition from other cabinets but ideally have a cable pass-through between them.
      • This section should still eventually list the "Depth from front rail to PDU (overall depth for hardware facing the front of the rack)" for each site.

Serial console

  • Opengear CM7148 dual PSU model
  • 48 port serial console server
  • We attach serial to every power strip and every bit of network kit (excluding the rack level mgmt switches)

Toolbag for Buildouts

When traveling to a buildout, the following tools are needed:

  • Phillips and Regular screwdrivers (hardware and rack repair/modifications)
  • Utility Knife (open boxes)
  • Label Maker (label hosts and cables)
  • Light Meter (measure xconnect light levels)
  • USB to Serial Adapter (early connection to PDU, SCS, and network devices)

Brazil Specific

  • Jenn to bring flat ribbon cable for PDU to PDU links
  • Rob to bring ulsfo light meter, label maker
  • Rob to bring usb to rj45/serial adapter & muti-bit megapro screwdriver.
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=SRE/Dc-operations/Common_Data_center_Specifications&oldid=2132782"