hadoop设置公平队列

时间:2023-03-09 19:35:47
hadoop设置公平队列

http://hadoop.apache.org/docs/r1.2.1/fair_scheduler.html

fair-scheduler.xml文档

<?xml version="1.0"?>
<allocations>
<pool name="test_pool">
<minMaps>5</minMaps>
<minReduces>5</minReduces>
<maxMaps>15</maxMaps>
<maxReduces>15</maxReduces>
<minSharePreemptionTimeout>300</minSharePreemptionTimeout>
</pool>
<user name="app">
<maxRunningJobs>5</maxRunningJobs>
</user>
<user name="ronnie.zhang">
<maxRunningJobs>5</maxRunningJobs>
</user>
<user name="dean.wu">
<maxRunningJobs>5</maxRunningJobs>
</user>
<userMaxJobsDefault>3</userMaxJobsDefault>
<fairSharePreemptionTimeout>600</fairSharePreemptionTimeout>
</allocations>

  mapred-site.xml文档中添加

   <property>
<name>mapred.jobtracker.taskScheduler</name>
<value>org.apache.hadoop.mapred.FairScheduler</value>
</property>

默认使用FIFO队列。

相关文章