Top "Hdfs" questions

Hadoop Distributed File System (HDFS) is the default file storage system used by Apache Hadoop.

Hadoop, how to compress mapper output but not the reducer output

I have a map-reduce java program in which I try to only compress the mapper output but not the reducer …

compression hadoop hdfs
How to get hadoop put to create directories if they don't exist

I have been using Cloudera's hadoop (0.20.2). With this version, if I put a file into the file system, but the …

hadoop hdfs cloudera put biginsights
Why does "hadoop fs -mkdir" fail with Permission Denied?

I am using Cloudera on a VM machine that I am playing around with. Unfortunately I am having issues copying …

hadoop hdfs cloudera
How to unzip .gz files in a new directory in hadoop?

I have a bunch of .gz files in a folder in hdfs. I want to unzip all of these .gz …

hadoop hdfs gzip
How to find Hadoop hdfs directory on my system?

How to find Hadoop HDFS directory on my system? I need this to run following command - hadoop dfs -copyFromLocal &…

linux azure hadoop hdfs hortonworks-data-platform
Differences between Amazon S3 and S3n in Hadoop

When I connected my Hadoop cluster to Amazon storage and downloaded files to HDFS, I found s3:// did not work. …

hadoop amazon-s3 hdfs
Append data to existing file in HDFS Java

I'm having trouble to append data to an existing file in HDFS. I want that if the file exists then …

java hadoop hdfs filewriter
How does Spark partition(ing) work on files in HDFS?

I'm working with Apache Spark on a Cluster using HDFS. As far as I understand, HDFS is distributing files on …

apache-spark hdfs
Spark iterate HDFS directory

I have a directory of directories on HDFS, and I want to iterate over the directories. Is there any easy …

hadoop hdfs apache-spark
Find port number where HDFS is listening

I want to access hdfs with fully qualified names such as : hadoop fs -ls hdfs://machine-name:8020/user I could also …

hadoop hdfs cloudera hortonworks-data-platform mapr