Analytics/Systems/Cluster/Ports

From Wikitech
Jump to: navigation, search

Ports

Hadoop

App Port Hosts Notes
Hadoop HDFS NameNode 8020 analytics1001,analytics1002
Hadoop HDFS over HTTP (HTTPFS) 14000 analytics1001,analytics1002
Hadoop HDFS NameNode HTTP UI 50070 analytics1001,analytics1002
Hadoop HDFS NameNode HTTPS UI 50470 analytics1001,analytics1002
Hadoop HDFS HA JournalNode 8485 analytics1028,analytics1035,analytics1052
Hadoop HDFS DataNode 50010 analytics1028-analytics1059
Hadoop HDFS DataNode HTTP UI 50075 analytics1028-analytics1059
Hadoop HDFS DataNode IPC 50020 analytics1028-analytics1059
Hadoop YARN ResourceManager 8032 analytics1001,analytics1002
Hadoop YARN ResourceManager Scheduler 8030 analytics1001,analytics1002
Hadoop YARN ResourceManager Tracker 8031 analytics1001,analytics1002
Hadoop YARN ResourceManager Admin 8033 analytics1001,analytics1002
Hadoop YARN ResourceManager HTTP UI 8088 analytics1001,analytics1002
Hadoop YARN NodeManager 8041 analytics1028-analytics1059
Hadoop YARN NodeManager localizer 8040 analytics1028-analytics1059
Hadoop YARN NodeManager HTTP UI 8042 analytics1028-analytics1059
Hadoop YARN App Masters random analytics1028-analytics1059
Spark Yarn Shuffle Service 7337 analytics1028-analytics1059
Hadoop MapReduce JobHistory Server 10020 analytics1001
Hadoop MapReduce Shuffle Port 13562 analytics1028-analytics1059
Hadoop MapReduce JobHistory HTTP UI 19888 analytics1001
Hive Server 10000 analytics1003
Hive MetaStore 9083 analytics1003
analytics-meta MySQL instance 3306 analytics1003
Zookeeper 2181 (client port) conf1001,conf1002,conf1003
Hue Server 8888 thorium
Oozie Server HTTP interface 11000 analytics1003
Oozie Server Admin Port 11001 analytics1003
Spark Local Client Driver HTTP UI 4040 and up, random stat1005 stat1004

Kafka

App Port Hosts Notes
Kafka Broker 9092 kafka1012,kafka1013,kafka1014,kafka1018,kafka1020,kafka1022

JMX

For use with jconsole, visualvm, jmxtrans etc.

App Port Hosts Notes
Hadoop Namenode(s) 9980 analytics1001,analytics1002
Hadoop Datanode 9981 analytics1028-analytics1059
Hadoop YARN ResourceManager 9983 analytics1001,analytics1002
Hadoop MapReduce History Server 9986 analytics1001
Hadoop YARN NodeManager 9984 analytics1028-analytics1059
Hadoop YARN ProxyServer 9985
Kafka 9999 kafka1012,kafka1013,kafka1014,kafka1018,kafka1020,kafka1022
Zookeeper 9998 conf1001,conf1002,conf1003
jmxtrans 2101 *
Druid Broker 9661 druid1001-1006
Druid Coordinator 9662 druid1001-1006
Druid Historical 9663 druid1001-1006
Druid Middle Manager 9664 druid1001-1006
Druid Overlord 9665 druid1001-1006

Prometheus JMX exporter

App Port Hosts Notes
Hadoop Namenode(s) 10080 analytics1001,analytics1002
Hadoop Datanode 51010 analytics1028-analytics1069
Hadoop YARN ResourceManager 10083 analytics1001,analytics1002
Hadoop MapReduce History Server 10086 analytics1001
Hadoop YARN NodeManager 8141 analytics1028-analytics1069
Kafka 7800 kafka-jumbo1001-1006
Druid Broker 8182 druid1001-1006
Druid Coordinator 8181 druid1001-1006
Druid Historical 8183 druid1001-1006
Druid Middle Manager 8191 druid1001-1006
Druid Overlord 8190 druid1001-1006

Druid

App Port Hosts Notes
Druid Coordinator 8081 druid100[123].eqiad.wmnet
Druid Broker 8082 druid100[123].eqiad.wmnet
Druid Historical 8083 druid100[123].eqiad.wmnet
Druid Overlord 8090 druid100[123].eqiad.wmnet
Druid MiddleManager 8091 druid100[123].eqiad.wmnet
Druid Indexing Peon Task 8100-9000 druid100[123].eqiad.wmnet ephemeral processes