How to copy the output of -text HDFS command into another file?

hadoop hdfs sequencefile
dewet picture dewet · Aug 22, 2014 · Viewed 7.4k times · Source

Is there any way we can copy text content of hdfs file into another file system using HDFS command:

 hadoop fs -text /user/dir1/abc.txt

Can I print the output of -text into another file by using -cat or any method ?:

 hadoop fs -cat /user/deepak/dir1/abc.txt

Answer

Aleksei Shestakov picture Aleksei Shestakov · Aug 22, 2014

As it's written in the documentation you can use hadoop fs -cp to copy files in hdfs. You can use hadoop fs -copyToLocal to copy files from hdfs to local file system. If you want to copy files from one hdfs to another then use DistCp tool.

Related questions

Write and read raw byte arrays in Spark - using Sequence File SequenceFile

How do you write RDD[Array[Byte]] to a file using Apache Spark and read it back again?

Read More
Advantages of Sequence file over hdfs textfile

What is the advantage of Hadoop Sequence File over HDFS flat file(Text)? In what way Sequence file is efficient? Small files can be combined and written into a sequence file, but the same can be done for a HDFS …

Read More
Spark - load CSV file as DataFrame?

I would like to read a CSV in spark and convert it as DataFrame and store it in HDFS with df.registerTempTable("table_name") I have tried: scala> val df = sqlContext.load("hdfs:///csv/file/dir/file.csv") Error …

Read More