Top "Yarn" questions

YARN (Yet Another Resource Negotiator) is a key component of second generation Apache Hadoop infrastructure.

YARN - Possible values for "Resource­Manager Web UI" port

Is the port for "Resource­Manager Web UI" for a yarn machine always set to 8088 or is it possible to …

rest port yarn dart-webui resourcemanager
Upload zip file using --archives option of spark-submit on yarn

I have a directory with some model files and my application has to access these models files in local file …

scala apache-spark zip yarn
Spark-submit / spark-shell > difference between yarn-client and yarn-cluster mode

I am running Spark with YARN. From the link: http://spark.apache.org/docs/latest/running-on-yarn.html I found explanation …

apache-spark yarn
YARN UNHEALTHY nodes

In our YARN cluster which is 80% full, we are seeing some of the yarn nodemanager's are marked as UNHEALTHY. after …

hadoop distributed-computing cloudera yarn cloudera-cdh
yarn is not honouring yarn.nodemanager.resource.cpu-vcores

I am using Hadoop-2.4.0 and my system configs are 24 cores, 96 GB RAM. I am using following configs mapreduce.map.cpu.…

hadoop mapreduce cloudera yarn hadoop2
Why does vcore always equal the number of nodes in Spark on YARN?

I have a Hadoop cluster with 5 nodes, each of which has 12 cores with 32GB memory. I use YARN as MapReduce …

apache-spark yarn
Spark executor logs on YARN

I'm launching a distributed Spark application in YARN client mode, on a Cloudera cluster. After some time I see some …

apache-spark cloudera yarn cloudera-manager
"Bad substitution" when submitting spark job to yarn-cluster

I am doing a smoke test against a yarn cluster using yarn-cluster as the master with the SparkPi example program. …

apache-spark yarn
Oozie shell action memory limit

We have an oozie workflow with a shell action that needs more memory than what a map task is given …

oozie yarn
Why does Yarn on EMR not allocate all nodes to running Spark jobs?

I'm running a job on Apache Spark on Amazon Elastic Map Reduce (EMR). Currently I'm running on emr-4.1.0 which includes …

apache-spark yarn emr amazon-emr elastic-map-reduce