Why does "hadoop fs -mkdir" fail with Permission Denied?

hadoop hdfs cloudera
AAA picture AAA · Mar 27, 2014 · Viewed 44.8k times · Source

I am using Cloudera on a VM machine that I am playing around with. Unfortunately I am having issues copying data to the HDFS, I am getting the following:

[cloudera@localhost ~]$ hadoop fs -mkdir input
mkdir: Permission denied: user=cloudera, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

I am not too concerned about security on this VM, is there anyway I can open up security more on HDFS?

Answer

xYan picture xYan · Nov 6, 2014

Using mkdir in hadoop needs the "hadoop file permissions". From your example you can see that hdfs is a user that has permissions to create folders. So if you run:

sudo -u hdfs hadoop fs -mkdir /import

then the import folder will be created. If you want to change the owner of this folder run:

sudo -u hdfs hadoop fs -chown new_user /import

Now the new_user can manipulate files inside the import folder

Related questions

How to get hadoop put to create directories if they don't exist

I have been using Cloudera's hadoop (0.20.2). With this version, if I put a file into the file system, but the directory structure did not exist, it automatically created the parent directories: So for example, if I had no directories in …

Read More
Find port number where HDFS is listening

I want to access hdfs with fully qualified names such as : hadoop fs -ls hdfs://machine-name:8020/user I could also simply access hdfs with hadoop fs -ls /user However, I am writing test cases that should work on different distributions(…

Read More
issue Running Spark Job on Yarn Cluster

I want to run my spark Job in Hadoop YARN cluster mode, and I am using the following command: spark-submit --master yarn-cluster --driver-memory 1g --executor-memory 1g --executor-cores 1 --class com.dc.analysis.jobs.AggregationJob sparkanalitic.jar param1 param2 param3 I am …

Read More