Hadoop pipe wordcount: Идентификатор задачи: попытка Статус: FAILED AttemptID: попытка Тайм-аут через 600 секунд

Мой Haddop - последняя версия 2.3.0. Я могу хорошо запустить приложение Hadoop Java. Но в приложении Hotsop pipe wordcount есть некоторые ошибки.

Это wordcount-simple.cc:

#include "Pipes.hh"
#include "TemplateFactory.hh"
#include "StringUtils.hh"

const std::string WORDCOUNT = "WORDCOUNT";
const std::string INPUT_WORDS = "INPUT_WORDS";
const std::string OUTPUT_WORDS = "OUTPUT_WORDS";

class WordCountMap: public HadoopPipes::Mapper {
public:
  HadoopPipes::TaskContext::Counter* inputWords;

  WordCountMap(HadoopPipes::TaskContext& context) {
    inputWords = context.getCounter(WORDCOUNT, INPUT_WORDS);
  }

  void map(HadoopPipes::MapContext& context) {
    std::vector<std::string> words =
      HadoopUtils::splitString(context.getInputValue(), " ");
      for(unsigned int i=0; i < words.size(); ++i) {
          context.emit(words[i], "1");
      }
    context.incrementCounter(inputWords, words.size());
  }
};

class WordCountReduce: public HadoopPipes::Reducer {
public:
  HadoopPipes::TaskContext::Counter* outputWords;

  WordCountReduce(HadoopPipes::TaskContext& context) {
    outputWords = context.getCounter(WORDCOUNT, OUTPUT_WORDS);
  }

  void reduce(HadoopPipes::ReduceContext& context) {
    int sum = 0;
    while (context.nextValue()) {
      sum += HadoopUtils::toInt(context.getInputValue());
    }
    context.emit(context.getInputKey(), HadoopUtils::toString(sum));
    context.incrementCounter(outputWords, 1);
  }
};

Ниже приведен Makefile:

HADOOP_INSTALL=/opt/hadoop

CC = g++
CCFLAGS = -I$(HADOOP_INSTALL)/include

wordcount :wordcount-simple.cc
        $(CC) $(CCFLAGS) $< -Wall -L$(HADOOP_INSTALL)/lib/native -lhadooppipes -lhadooputils -lpthread -lcrypto -g -O2 -o $@

Он компилируется ОК. Затем я загружаю некоторые данные для тестирования. Моя команда запуска:

hadoop pipes -D hadoop.pipes.java.recordreader=true -D hadoop.pipes.java.recordwriter=true -D mapred.job.name=wordcount -input /data/wc_in -output /data/wc_out -program /bin/wordcount

но возникает некоторая ошибка:

14/04/03 23:59:48 INFO client.RMProxy: Connecting to ResourceManager at /192.168.1.137:8032
14/04/03 23:59:49 INFO client.RMProxy: Connecting to ResourceManager at /192.168.1.137:8032
14/04/03 23:59:50 WARN mapreduce.JobSubmitter: No job jar file set. User classes may not be found. See Job or Job#setJar(String). 
14/04/03 23:59:50 INFO mapred.FileInputFormat: Total input paths to process : 2
14/04/03 23:59:51 INFO mapreduce.JobSubmitter: number of splits:2
14/04/03 23:59:51 INFO Configuration.deprecation: hadoop.pipes.java.recordreader is deprecated. Instead, use mapreduce.pipes.isjavarecordreader
14/04/03 23:59:51 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
14/04/03 23:59:51 INFO Configuration.deprecation: hadoop.pipes.java.recordwriter is deprecated. Instead, use mapreduce.pipes.isjavarecordwriter
14/04/03 23:59:52 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1396578697573_0004
14/04/03 23:59:52 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
14/04/03 23:59:53 INFO impl.YarnClientImpl: Submitted application application_1396578697573_0004
14/04/03 23:59:53 INFO mapreduce.Job: The url to track the job: 
14/04/03 23:59:53 INFO mapreduce.Job: Running job: job_1396578697573_0004
14/04/04 00:00:26 INFO mapreduce.Job: Job job_1396578697573_0004 running in uber mode : false

**14/04/04 00:00:26 INFO mapreduce.Job: map 0% reduce 0%
14/04/04 00:10:53 INFO mapreduce.Job: map 100% reduce 0%
14/04/04 00:10:53 INFO mapreduce.Job: Task Id : attempt_1396578697573_0004_m_000001_0, Status : FAILED
AttemptID:attempt_1396578697573_0004_m_000001_0 Timed out after 600 secs
14/04/04 00:10:54 INFO mapreduce.Job: Task Id : attempt_1396578697573_0004_m_000000_0, Status : FAILED
AttemptID:attempt_1396578697573_0004_m_000000_0 Timed out after 600 secs
14/04/04 00:10:55 INFO mapreduce.Job: map 0% reduce 0%
14/04/04 00:21:23 INFO mapreduce.Job: map 100% reduce 0%
14/04/04 00:21:24 INFO mapreduce.Job: Task Id : attempt_1396578697573_0004_m_000000_1, Status : FAILED
AttemptID:attempt_1396578697573_0004_m_000000_1 Timed out after 600 secs
14/04/04 00:21:24 INFO mapreduce.Job: Task Id : attempt_1396578697573_0004_m_000001_1, Status : FAILED
AttemptID:attempt_1396578697573_0004_m_000001_1 Timed out after 600 secs
14/04/04 00:21:25 INFO mapreduce.Job: map 0% reduce 0%
14/04/04 00:31:53 INFO mapreduce.Job: Task Id : attempt_1396578697573_0004_m_000000_2, Status : FAILED
AttemptID:attempt_1396578697573_0004_m_000000_2 Timed out after 600 secs
14/04/04 00:31:53 INFO mapreduce.Job: Task Id : attempt_1396578697573_0004_m_000001_2, Status : FAILED
AttemptID:attempt_1396578697573_0004_m_000001_2 Timed out after 600 secs
14/04/04 00:42:24 INFO mapreduce.Job: map 100% reduce 0%
14/04/04 00:42:25 INFO mapreduce.Job: map 100% reduce 100%
14/04/04 00:42:26 INFO mapreduce.Job: Job job_1396578697573_0004 failed with state FAILED due to: Task failed task_1396578697573_0004_m_000000

Job failed as tasks failed. failedMaps:1 failedReduces:0
14/04/04 00:42:27 INFO mapreduce.Job: Counters: 9
 Job Counters
  Failed map tasks=8
  Launched map tasks=8
  Other local map tasks=6
  Data-local map tasks=2
  Total time spent by all maps in occupied slots (ms)=5017539
  Total time spent by all reduces in occupied slots (ms)=0
  Total time spent by all map tasks (ms)=5017539
  Total vcore-seconds taken by all map tasks=5017539
  Total megabyte-seconds taken by all map tasks=5137959936

Exception in thread "main" java.io.IOException: Job failed!
 at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
 at org.apache.hadoop.mapred.pipes.Submitter.runJob(Submitter.java:264)
 at org.apache.hadoop.mapred.pipes.Submitter.run(Submitter.java:503)
 at org.apache.hadoop.mapred.pipes.Submitter.main(Submitter.java:518)**

Он длится долго, но не удался. Что-то не так с моими настройками? Но я хорошо запускаю приложения без труб, никаких ошибок. поэтому я не думаю, что проблема с моей окружающей средой.

Кроме того, никаких ошибок в записи именования (datanode) не регистрируется. Поэтому я удивляюсь, почему. Заранее спасибо.