Container exited with a non-zero exit code 1 error during mapreduce task

Question

On executing jar in hadoop, I get the following error:

16/11/04 18:32:59 INFO mapreduce.Job: Task Id : attempt_1478261728730_0005_m_000000_2, Status : FAILED
Exception from container-launch.
Container id: container_1478261728730_0005_01_000004
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1

16/11/04 18:33:09 INFO mapreduce.Job:  map 100% reduce 0%

This is application log:

Native code library failed to load.
java.lang.UnsatisfiedLinkError: no opencv_java2411 in java.library.pathopencv_java2411

I don't know what it mean, can anybody help with this please?

Answer 1

You are missing opencv on your cluster nodes. See here for all the details on how to handle this.

Long story short though, you need to install opencv on your executors. You cannot really compile it into your job's .jar in a portable way since it's C and not Java code.

Update: Note that the environment on your Hadoop executors is set by your hadoop-env.sh. So it needs to contain a line like:

JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:/etc/opencv/lib