Top "Yarn" questions

YARN (Yet Another Resource Negotiator) is a key component of second generation Apache Hadoop infrastructure.

Spark Kill Running Application

I have a running Spark application where it occupies all the cores where my other applications won't be allocated any …

apache-spark yarn pyspark
Container is running beyond memory limits

In Hadoop v1, I have assigned each 7 mapper and reducer slot with size of 1GB, my mappers & reducers runs …

hadoop mapreduce yarn mrv2
Apache Spark: The number of cores vs. the number of executors

I'm trying to understand the relationship of the number of cores and the number of executors when running a Spark …

hadoop apache-spark yarn
Where are logs in Spark on YARN?

I'm new to spark. Now I can run spark 0.9.1 on yarn (2.0.0-cdh4.2.1). But there is no log after execution. The …

hadoop logging apache-spark cloudera yarn
Hadoop: Connecting to ResourceManager failed

After installing hadoop 2.2 and trying to launch pipes example ive got the folowing error (the same error shows up after …

hadoop yarn
Where does Hadoop store the logs of YARN applications?

I run the basic example of Hortonworks' yarn application example. The application fails and I want to read the logs …

logging hadoop yarn
How to log using log4j to local file system inside a Spark application that runs on YARN?

I'm building an Apache Spark Streaming application and cannot make it log to a file on the local filesystem when …

logging log4j apache-spark yarn
What is yarn-client mode in Spark?

Apache Spark has recently updated the version to 0.8.1, in which yarn-client mode is available. My question is, what does yarn-client …

yarn apache-spark
What is the relation between 'mapreduce.map.memory.mb' and 'mapred.map.child.java.opts' in Apache Hadoop YARN?

I would like to know the relation between the mapreduce.map.memory.mb and mapred.map.child.java.opts parameters. …

apache hadoop configuration yarn heap-size
What is a container in YARN?

What is a container in YARN? Is it same as the child JVM in which the tasks on the nodemanager …

hadoop mapreduce yarn