Top "Hadoop2" questions

yarn is not honouring yarn.nodemanager.resource.cpu-vcores

I am using Hadoop-2.4.0 and my system configs are 24 cores, 96 GB RAM. I am using following configs mapreduce.map.cpu.…

hadoop mapreduce cloudera yarn hadoop2

Merging small files in hadoop

I have a directory (Final Dir) in HDFS in which some files(ex :10 mb) are loading every minute. After some …

hadoop mapreduce hive hdfs hadoop2

Spark/Yarn: File does not exist on HDFS

I have a Hadoop/Yarn cluster setup on AWS, I have one master and 3 slaves. I have verified I have 3 …

hadoop apache-spark pyspark yarn hadoop2

Minimum system requirements for running a Hadoop Cluster with High Availability

From what I understand for High availability in hadoop we need one Name Node and one Standby Node,Network shared …

hadoop hdfs hadoop2 high-availability

What does virtual core in YARN vcore mean?

Yarn is using the concept of virtual core to manage CPU resources. I would ask what's the benefit to use …

hadoop yarn hadoop2

Getting java.lang.IllegalArgumentException: requirement failed while calling Sparks MLLIB StreamingKMeans from java application

I am new to Spark and MLlib and I'm trying to call StreamingKMeans from my java application and I get …

java apache-spark bigdata hadoop2 spark-streaming

Using reserved words in Hive

I'm migrating data to Hive 1.2, and I realized that, by default, I'm no longer allowed to use reserved words as …

hadoop hive hiveql hadoop2 hive-configuration