hadoop copy a local file system folder to HDFS

hadoop hdfs
Tariq picture Tariq · Jan 29, 2015 · Viewed 214.5k times · Source

I need to copy a folder from local file system to HDFS. I could not find any example of moving a folder(including its all subfolders) to HDFS

$ hadoop fs -copyFromLocal /home/ubuntu/Source-Folder-To-Copy HDFS-URI

Answer

Ashrith picture Ashrith · Jan 29, 2015

You could try:

hadoop fs -put /path/in/linux /hdfs/path

or even

hadoop fs -copyFromLocal /path/in/linux /hdfs/path

By default both put and copyFromLocal would upload directories recursively to HDFS.

Related questions

Spark - load CSV file as DataFrame?

I would like to read a CSV in spark and convert it as DataFrame and store it in HDFS with df.registerTempTable("table_name") I have tried: scala> val df = sqlContext.load("hdfs:///csv/file/dir/file.csv") Error …

Read More
How to copy file from HDFS to the local file system

How to copy file from HDFS to the local file system . There is no physical location of a file under the file , not even directory . how can i moved them to my local for further validations.i am tried through …

Read More
Name node is in safe mode. Not able to leave

root# bin/hadoop fs -mkdir t mkdir: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /user/root/t. Name node is in safe mode. not able to create anything in hdfs I did root# bin/hadoop fs -safemode …

Read More