Hadoop java.io.IOException: Mkdirs failed to create /some/path

hadoop ioexception microsoft-distributed-file-system
alien01 picture alien01 · May 9, 2012 · Viewed 56.1k times · Source

When I try to run my Job I am getting the following exception:

Exception in thread "main" java.io.IOException: Mkdirs failed to create /some/path
    at org.apache.hadoop.util.RunJar.ensureDirectory(RunJar.java:106)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:150)

Where the /some/path is hadoop.tmp.dir. However when I issue the dfs -ls cmd on /some/path I can see that it exists and the dataset file is present (was copied before lunching the job). Also the path is correctly defined in hadoop configs. Any suggestions will be appreciated. I am using hadoop 0.21.

Answer

Todd Nemet picture Todd Nemet · Jul 8, 2012

Just ran into this problem running mahout from CDH4 in standalone mode in my MacBook Air.

The issue is that a /tmp/hadoop-xxx/xxx/LICENSE file and a /tmp/hadoop-xxx/xxx/license directory are being created on a case-insensitive file system when unjarring the mahout jobs.

I was able to workaround this by deleting META-INF/LICENSE from the jar file like this:

zip -d mahout-examples-0.6-cdh4.0.0-job.jar META-INF/LICENSE

and then verified it with

jar tvf mahout-examples-0.6-cdh4.0.0-job.jar | grep -i license

Hope this helps!

Related questions

hadoop java.io.IOException: while running namenode -format

I ran namenode -format.This is my output. I tried changing the file permissions chmod 777 hadoop. I believe this line is the error ERROR namenode.NameNode: java.io.IOException: Cannot create directory /your/path/to/hadoop/tmp/dir/hadoop-hadoop/dfs/…

Read More
Hadoop IOException failure to login

I'm pretty new to Hadoop. However, I've been able to successfully setup hadoop 2.7.3 with Java 7 in the cluster mode on my servers. Everything works totally fine. But then, when I try to switch to Java 8 and start dfs, there is …

Read More
Hadoop "Unable to load native-hadoop library for your platform" warning

I'm currently configuring hadoop on a server running CentOs. When I run start-dfs.sh or stop-dfs.sh, I get the following error: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable I'm running …

Read More