Ha-Federation-hdfs +Yarn集群部署方式

时间:2023-03-08 18:55:51

经过一下午的尝试,终于把这个集群的搭建好了,搭完感觉也没有太大的必要,就当是学习了吧,为之后搭建真实环境做基础。

以下搭建的是一个Ha-Federation-hdfs+Yarn的集群部署。

  首先讲一下我的配置:

  四个节点上的启动的分别是:

  1.qiang117:active namenode,

  2.qiang118 standby namenode ,journalnode,datanode

  3.qiang119 active namenode    ,journalnode,datanode

  4.qiang120 standby namenode ,journalnode,datanode

  这样做纯粹是因为电脑hold不住那么虚拟机了,其实这里所有的节点都应该在不同的服务器上。简单的说,就是117和119做active namenode,118和120做standby namenode,在118.119.120上分别放datanode和journalnode。

此处省略一万字,各种配置好之后。。遇到的问题和记录如下:

1.启动 journalnode,这个journalnode话说我也不是太明白他是干嘛的~~,后续研究吧。在各个节点上启动journalnode:

[qiang@qiang118 hadoop-2.6.]$ sbin/hadoop-daemon.sh start journalnode
starting journalnode, logging to /home/qiang/hadoop-2.6./logs/hadoop-qiang-journalnode-qiang118.qiang.out
[qiang@qiang118 hadoop-2.6.]$ jps
JournalNode
Jps

2. 格式化namenode时报错:(最后查出来是没有关防火墙。。。免密码登陆不代表不用关防火墙)

// :: INFO ipc.Client: Retrying connect to server: qiang119/192.168.75.119:. Already tried  time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=, sleepTime= MILLISECONDS)
// :: INFO ipc.Client: Retrying connect to server: qiang118/192.168.75.118:. Already tried time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=, sleepTime= MILLISECONDS)
// :: INFO ipc.Client: Retrying connect to server: qiang120/192.168.75.120:. Already tried time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=, sleepTime= MILLISECONDS)
// :: INFO ipc.Client: Retrying connect to server: qiang119/192.168.75.119:. Already tried time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=, sleepTime= MILLISECONDS)
// :: WARN namenode.NameNode: Encountered exception during format:
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if JNs are ready for formatting. exceptions thrown:
192.168.75.120:: No Route to Host from 43.49.49.59.broad.ty.sx.dynamic.163data.com.cn/59.49.49.43 to qiang120: failed on socket timeout exception: java.net.NoRouteToHostException: No route to host; For more details see: http://wiki.apache.org/hadoop/NoRouteToHost
192.168.75.119:: No Route to Host from 43.49.49.59.broad.ty.sx.dynamic.163data.com.cn/59.49.49.43 to qiang119: failed on socket timeout exception: java.net.NoRouteToHostException: No route to host; For more details see: http://wiki.apache.org/hadoop/NoRouteToHost
at org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:)
at org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:)
at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:)
at org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:)
at org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:)
// :: INFO ipc.Client: Retrying connect to server: qiang118/192.168.75.118:. Already tried time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=, sleepTime= MILLISECONDS)
// :: FATAL namenode.NameNode: Failed to start namenode.
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if JNs are ready for formatting. exceptions thrown:
192.168.75.120:: No Route to Host from 43.49.49.59.broad.ty.sx.dynamic.163data.com.cn/59.49.49.43 to qiang120: failed on socket timeout exception: java.net.NoRouteToHostException: No route to host; For more details see: http://wiki.apache.org/hadoop/NoRouteToHost
192.168.75.119:: No Route to Host from 43.49.49.59.broad.ty.sx.dynamic.163data.com.cn/59.49.49.43 to qiang119: failed on socket timeout exception: java.net.NoRouteToHostException: No route to host; For more details see: http://wiki.apache.org/hadoop/NoRouteToHost
at org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:)
at org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:)
at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:)
at org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:)
at org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:)
// :: INFO util.ExitUtil: Exiting with status
// :: INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at 43.49.49.59.broad.ty.sx.dynamic.163data.com.cn/59.49.49.43

格式化成功!

[qiang@qiang117 hadoop-2.6.]$ bin/hdfs namenode -format -clusterId hadoop-cluster
// :: INFO namenode.FSNamesystem: Append Enabled: true
// :: INFO util.GSet: Computing capacity for map INodeMap
// :: INFO util.GSet: VM type = -bit
// :: INFO util.GSet: 1.0% max memory MB = 8.9 MB
// :: INFO util.GSet: capacity = ^ = entries
// :: INFO namenode.NameNode: Caching file names occuring more than times
// :: INFO util.GSet: Computing capacity for map cachedBlocks
// :: INFO util.GSet: VM type = -bit
// :: INFO util.GSet: 0.25% max memory MB = 2.2 MB
// :: INFO util.GSet: capacity = ^ = entries
// :: INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
// :: INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes =
// :: INFO namenode.FSNamesystem: dfs.namenode.safemode.extension =
// :: INFO namenode.FSNamesystem: Retry cache on namenode is enabled
// :: INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is millis
// :: INFO util.GSet: Computing capacity for map NameNodeRetryCache
// :: INFO util.GSet: VM type = -bit
// :: INFO util.GSet: 0.029999999329447746% max memory MB = 273.1 KB
// :: INFO util.GSet: capacity = ^ = entries
// :: INFO namenode.NNConf: ACLs enabled? false
// :: INFO namenode.NNConf: XAttrs enabled? true
// :: INFO namenode.NNConf: Maximum size of an xattr:
// :: INFO namenode.FSImage: Allocated new BlockPoolId: BP--192.168.75.117-
// :: INFO common.Storage: Storage directory /home/qiang/hadoop/hdfs/name has been successfully formatted.
// :: INFO namenode.NNStorageRetentionManager: Going to retain images with txid >=
// :: INFO util.ExitUtil: Exiting with status
// :: INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at qiang117/192.168.75.117
************************************************************/

3.开启namenode:

[qiang@qiang117 hadoop-2.6.]$ sbin/hadoop-daemon.sh start namenode
starting namenode, logging to /home/qiang/hadoop-2.6./logs/hadoop-qiang-namenode-qiang117.out
[qiang@qiang117 hadoop-2.6.]$ jps
NameNode
Jps

4.格式化standby namenode

[qiang@qiang119 hadoop-2.6.]$ bin/hdfs namenode -bootstrapStandby
// :: INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = qiang119/192.168.75.119
STARTUP_MSG: args = [-bootstrapStandby]
STARTUP_MSG: version = 2.6.0
.....
.....
STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z
STARTUP_MSG: java = 1.8.0_51
************************************************************/
// :: INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
// :: INFO namenode.NameNode: createNameNode [-bootstrapStandby]
=====================================================
About to bootstrap Standby ID nn2 from:
Nameservice ID: hadoop-cluster1
Other Namenode ID: nn1
Other NN's HTTP address: http://qiang117:50070
Other NN's IPC address: qiang117/192.168.75.117:8020
Namespace ID:
Block pool ID: BP--192.168.75.117-
Cluster ID: hadoop-cluster
Layout version: -
=====================================================
// :: INFO common.Storage: Storage directory /home/qiang/hadoop/hdfs/name has been successfully formatted.
// :: INFO namenode.TransferFsImage: Opening connection to http://qiang117:50070/imagetransfer?getimage=1&txid=0&storageInfo=-60:1244139539:0:hadoop-cluster
// :: INFO namenode.TransferFsImage: Image Transfer timeout configured to milliseconds
// :: INFO namenode.TransferFsImage: Transfer took .01s at 0.00 KB/s
// :: INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size bytes.
// :: INFO util.ExitUtil: Exiting with status
// :: INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at qiang119/192.168.75.119
************************************************************/

5.开启standby namenode

[qiang@qiang119 hadoop-2.6.]$ sbin/hadoop-daemon.sh start namenode
starting namenode, logging to /home/qiang/hadoop-2.6./logs/hadoop-qiang-namenode-qiang119.out
[qiang@qiang119 hadoop-2.6.]$ jps
JournalNode
NameNode
Jps

在web上 打开以后二个显示都是standy状态:

Ha-Federation-hdfs +Yarn集群部署方式

Ha-Federation-hdfs +Yarn集群部署方式

使用这个命令将nn1切换为active状态:

bin/hdfs haadmin -ns hadoop-cluster1 -transitionToActive nn1

Ha-Federation-hdfs +Yarn集群部署方式

另外两个一样的道理:

Ha-Federation-hdfs +Yarn集群部署方式

Ha-Federation-hdfs +Yarn集群部署方式

开启所有的datanode,这里是在只有配置好ssh免密码登录的情况下才能使用。可以参考:http://www.cnblogs.com/qiangweikang/p/4740936.html

[qiang@qiang117 hadoop-2.6.]$ sbin/hadoop-daemons.sh start datanode

开了仨,就是之前预设好的192.168.1.118,192.168.1.119和192.168.1.120

Ha-Federation-hdfs +Yarn集群部署方式

启动yarn

[qiang@qiang117 hadoop-2.6.]$ sbin/start-yarn.sh 
starting yarn daemons
starting resourcemanager, logging to /home/qiang/hadoop-2.6./logs/yarn-qiang-resourcemanager-qiang117.out
qiang118: nodemanager running as process . Stop it first.
qiang120: nodemanager running as process . Stop it first.
qiang119: nodemanager running as process . Stop it first.
[qiang@qiang117 hadoop-2.6.]$ jps
NameNode
Jps
ResourceManager

也是可以看到有三个datanode

Ha-Federation-hdfs +Yarn集群部署方式

最后总结一下吧...... 自学大数据的话,有一个简单的部署就足够了,能够让你写好的程序放入hdfs中跑就可以了,这样的集群应该是在最后,或者需要的时候再去详细的做研究,抓紧进入之后的阶段吧~~