hadoop问题锦集(一):mapreduce不能在集群中运行

时间:2023-02-28 22:56:45

问题详细:

  搭建类hadoop的集群集群环境,查看hadoop dfsadmin -report 也有datanode节点.在master:50070上也有.

  然而在eclipse环境上运行mapreduce的程序时,却发现每次都是直接在本地上跑,并没有在集群上运行程序!

解决办法:

  =>mapred-site.xml里面mapreduce.jobtracker.address 这个属性没有配置,

    那么程序就会在导入的hadoop-mapreduce-client-core-2.5.2.jar包下去读mapred-default.xml的默认配置:local!  

<property>
<name>mapreduce.jobtracker.address</name>
<value>local</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>

  将上面的local改为你的namenode:port即可:

  例如(我的配置):

  

<property>
<name>mapreduce.jobtracker.address</name>
<value>master:9000</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>

  当然,这个修改最好是在你的hadoop_home/etc/hadoop/mapred-site.xml(conf下)修改.全局环境下都生效.