How to find Hadoop hdfs directory on my system?

N.. picture N.. · Apr 2, 2016 · Viewed 43.6k times · Source

How to find Hadoop HDFS directory on my system? I need this to run following command -

hadoop dfs -copyFromLocal <local-dir> <hdfs-dir>

In this command I don't knon my hdfs-dir.

Not sure if its helpful or not but I ran following command and got this output -

 hdfs dfs -ls
-rw-r--r--   3 popeye hdfs  127162942 2016-04-01 19:47 .

In hdfs-site.xml, I found following entry -

<property>
      <name>dfs.datanode.data.dir</name>
      <value>/hadoop/hdfs/data</value>
      <final>true</final>
</property>

I tried to run following command but it gives error -

[root@sandbox try]# hdfs dfs -copyFromLocal 1987.csv /hadoop/hdfs/data
copyFromLocal: `/hadoop/hdfs/data': No such file or directory

FYI - I am doing all this on hortonworks sandbox on azure server.

Answer

BruceWayne picture BruceWayne · Apr 3, 2016

Your approach is wrong or may be understanding is wrong

dfs.datanode.data.dir, is where you want to store your data blocks

If you type hdfs dfs -ls / you will get list of directories in hdfs. Then you can transfer files from local to hdfs using -copyFromLocal or -put to a particular directory or using -mkdir you can create new directory

Refer below link for more information

http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html