Top "Yarn" questions

Hadoop Error - All data nodes are aborting

I am using Hadoop 2.3.0 version. Sometimes when I execute the Map reduce job, the below errors will get displayed. 14/08/10 12:14:59 INFO …

hadoop mapreduce hdfs yarn hadoop2

Permission Denied error while running start-dfs.sh

I am getting this error while performing start-dfs.sh Starting namenodes on [localhost] pdsh@Gaurav: localhost: rcmd: socket: Permission denied …

sockets hadoop hdfs yarn hadoop2

Standalone Manager Vs. Yarn Vs. Mesos

On 3 node Spark/Hadoop cluster which scheduler(Manager) will work efficiently? Currently I am using Standalone Manager, but for each …

hadoop apache-spark yarn mesos

Why does a JVM report more committed memory than the linux process resident set size?

When running a Java app (in YARN) with native memory tracking enabled (-XX:NativeMemoryTracking=detail see https://docs.oracle.com/…

linux memory jvm yarn

WARN cluster.YarnScheduler: Initial job has not accepted any resources

Any spark jobs that I run will fail with the following error message 17/06/16 11:10:43 WARN cluster.YarnScheduler: Initial job has not …

apache-spark pyspark yarn taskscheduler

What is Memory reserved on Yarn

I managed to launch a spark application on Yarn. However emory usage is kind of weird as you can see …

hadoop apache-spark yarn hadoop2

/bin/bash: /bin/java: No such file or directory error in Yarn apps in MacOS

I was trying to run a simple wordcount MapReduce Program using Java 1.7 SDK and Hadoop2.7.1 on Mac OS X EL …

java macos hadoop mapreduce yarn

What is the correct way to start/stop spark streaming jobs in yarn?

I have been experimenting and googling for many hours, with no luck. I have a spark streaming app that runs …

hadoop apache-spark spark-streaming yarn cloudera

What additional benefit does Yarn bring to the existing map reduce?

Yarn differs in its infrastructure layer from the original map reduce architecture in the following way: In YARN, the job …

hadoop mapreduce yarn

Setting YARN queue in PySpark

When creating a Spark context in PySpark, I typically use the following code: conf = (SparkConf().setMaster("yarn-client").setAppName(appname) .set("…

hadoop apache-spark pyspark yarn