当前位置: 动力学知识库 > 问答 > 编程问答 >

Error in Hadoop MapReduce

问题描述:

When I run a mapreduce program using Hadoop, I get the following error.

10/01/18 10:52:48 INFO mapred.JobClient: Task Id : attempt_201001181020_0002_m_000014_0, Status : FAILED

java.io.IOException: Task process exit with nonzero status of 1.

at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418)

10/01/18 10:52:48 WARN mapred.JobClient: Error reading task outputhttp://ubuntu.ubuntu-domain:50060/tasklog?plaintext=true&taskid=attempt_201001181020_0002_m_000014_0&filter=stdout

10/01/18 10:52:48 WARN mapred.JobClient: Error reading task outputhttp://ubuntu.ubuntu-domain:50060/tasklog?plaintext=true&taskid=attempt_201001181020_0002_m_000014_0&filter=stderr

What is this error about?

网友答案:

One reason Hadoop produces this error is when the directory containing the log files becomes too full. This is a limit of the Ext3 Filesystem which only allows a maximum of 32000 links per inode.

Check how full your logs directory is in hadoop/userlogs

A simple test for this problem is to just try and create a directory from the command-line for example: $ mkdir hadoop/userlogs/testdir

If you have too many directories in userlogs the OS should fail to create the directory and report there are too many.

网友答案:

I was having the same issue when I run out of space on disk with log directory.

网友答案:

Another cause can be, JVM Error when you try to allocate some dedicated space to JVM and it is not present on your machine.

sample code:
conf.set("mapred.child.java.opts", "-Xmx4096m");

Error message:
Error occurred during initialization of VM
Could not reserve enough space for object heap

Solution: Replace -Xmx with dedicated memory value that you can provide to JVM on your machine(e.g. "-Xmx1024m")

网友答案:

Increase your ulimit to unlimited. or alternate solution reduce the allocated memory.

网友答案:

If you create a runnable jar file in eclipse, it gives that error on hadoop system. You should extract runnable part. That solved my problem.

分享给朋友:
您可能感兴趣的文章:
随机阅读: