Analytics/Systems/Cluster/Hardware

From Wikitech
Jump to: navigation, search

Information about the Analytics Cluster hardware and infrastructure.


Docs

Hardware Specs

By Host

Type # Hosts Processors Memory Disk
Dell PowerEdge R420 3 analytics1001, analytics1002, analytics1003 16 core Intel(R) Xeon(R) CPU E5-2450 v2 @ 2.50GHz 64G RAM 4 * 2T
Dell PowerEdge R720 12 aqs100{1,2,3},kafka10{12,13,14,18,20,22} 12 core EW-2620 @ 2.00GHz 48G RAM 24T = 12 * 2T
Dell PowerEdge R720 29 analytics1028 - analytics1057 12 core EW-2620 @ 2.00GHz 64G RAM 48T = 12 * 4T, + 2 SSDs

By Machine Type

Type Dell PowerEdge R420 Dell PowerEdge R720 Dell PowerEdge R720
Hosts analytics1001, analytics1002, analytics1003 aqs100{1,2,3},kafka10{12,13,14,18,20,22} analytics1028-analytics1057
Processors 16 core Intel(R) Xeon(R) CPU E5-2450 v2 @ 2.50GHz 12 core EW-2620 @ 2.00GHz 12 core EW-2620 @ 2.00GHz
Memory 64G RAM 48G RAM 64G RAM
Disk 4 * 2T 24T = 12 * 2T 48T = 12 * 4T + 2 SSDs

Roles

Hosts # Type Role
analytics1001 2 R420 Primary NameNode, Hadoop master node.
analytics1002 2 R420 Standby NameNode
analytics1003 1 R420 Hive, Oozie, MySQL, Camus crons, other crons, HDFS balancer
analytics1028-analytics1057 30 R720 Hadoop Worker Nodes (DataNodes)
kafka1012,kafka1013,kafka1014,kafka1018,kafka1020,kafka1022 6 R720 Kafka Brokers
thorium 1 Analytics webservice box. Hue.
aqs1001-aqs1003 3 R720 Analytics Query Service (RESTBase + Cassandra)

Notes:

- As of June 2015, analytics102[345] which were Zookeeper servers in the Analytics Cluster have been moved out into the general purpose production network and renamed as conf100[123]. These are still Zookeeper servers, but will also host etcd.

- In August 2015, analytics10xx kafka brokers have been renamed to kafka10xx with the numbers (e.g. analytics1012 -> kafka1012).

- In October 2015, analytics1011,1016,1019 were repurposed as aqs1001,1002,1003.