How to delete files from the HDFS?

hadoop hdfs hortonworks-data-platform
serg picture serg · Dec 7, 2015 · Viewed 84.8k times · Source

I just downloaded Hortonworks sandbox VM, inside it there are Hadoop with the version 2.7.1. I adding some files by using the

hadoop fs -put /hw1/* /hw1

...command. After it I am deleting the added files, by the

hadoop fs -rm /hw1/*

...command, and after it cleaning the recycle bin, by the

hadoop fs -expunge

...command. But the DFS Remaining space not changed after recyle bin cleaned. Even I can see that the data was truly deleted from the /hw1/ and the recyle bin. I have the fs.trash.interval parameter = 1.

Actually I can find all my data split in chunks in the /hadoop/hdfs/data/current/BP-2048114545-10.0.2.15-1445949559569/current/finalized/subdir0/subdir2 folder, and this is really surprises me, because I expect them to be deleted.

So my question how to delete the data the way that they really will be deleted? After few adding and deletion I got exhausted free space.

Answer

BruceWayne picture BruceWayne · Dec 8, 2015

Try hadoop fs -rm -R URI

-R option deletes the directory and any content under it recursively.

Related questions

How to find Hadoop hdfs directory on my system?

How to find Hadoop HDFS directory on my system? I need this to run following command - hadoop dfs -copyFromLocal <local-dir> <hdfs-dir> In this command I don't knon my hdfs-dir. Not sure if its helpful or …

Read More
Find port number where HDFS is listening

I want to access hdfs with fully qualified names such as : hadoop fs -ls hdfs://machine-name:8020/user I could also simply access hdfs with hadoop fs -ls /user However, I am writing test cases that should work on different distributions(…

Read More
hadoop user file permissions

i have a problem in setting hadoop file permissions in hortonworks and cloudera. My requirement is: 1. create a new user with new group 2. create user directory in hdfs ( ex. /user/myuser ) 3. Now this folder ( in this case /user/myuser ) must …

Read More