Top "Yarn" questions

YARN (Yet Another Resource Negotiator) is a key component of second generation Apache Hadoop infrastructure.

Hadoop YARN: Get a list of available queues

Is there a way to get a list of all available YARN queues from the command line, without resorting to …

hadoop yarn
Apache Hadoop Yarn - Underutilization of cores

No matter how much I tinker with the settings in yarn-site.xml i.e using all of the below options …

hadoop apache-spark yarn resourcemanager
Spark/Yarn: File does not exist on HDFS

I have a Hadoop/Yarn cluster setup on AWS, I have one master and 3 slaves. I have verified I have 3 …

hadoop apache-spark pyspark yarn hadoop2
Spark: get number of cluster cores programmatically

I run my spark application in yarn cluster. In my code I use number available cores of queue for creating …

java apache-spark dataset yarn core
YARN Application exited with exitCode: -1000 Not able to initialize user directories

I am getting: Application application_1427711869990_0001 failed 2 times due to AM Container for appattempt_1427711869990_0001_000002 exited with exitCode: -1000 due to: Not …

cloudera yarn
Apache Spark on YARN: Large number of input data files (combine multiple input files in spark)

A help for the implementation best practice is needed. The operating environment is as follows: Log data file arrives irregularly. …

hadoop apache-spark yarn
What does virtual core in YARN vcore mean?

Yarn is using the concept of virtual core to manage CPU resources. I would ask what's the benefit to use …

hadoop yarn hadoop2
How to set Spark application exit status?

I'm writing a spark application and run it using spark-submit shell script (using yarn-cluster/yarn-client) As I see now, exit …

apache-spark yarn exit-code spark-submit
hadoop - How to kill a TEZ job started by hive?

Below is what I can find. But the problem is if we reuse jdbc hive session all the hive queries …

hadoop hive yarn tez bigdata
Using typesafe config with Spark on Yarn

I have a Spark job that reads data from a configuration file. This file is a typesafe config file. The …

scala apache-spark yarn typesafe-config