Top "Yarn" questions

Spark on YARN resource manager: Relation between YARN Containers and Spark Executors

I'm new to Spark on YARN and don't understand the relation between the YARN Containers and the Spark Executors. I …

apache-spark containers yarn hortonworks-data-platform executor

How to kill an application from the ResourceManager Web UI

Is there a way of killing an application from the RM web UI instead of running yarn application -kill?

hadoop yarn resourcemanager

How to set DEBUG log level for ResourceManager?

I am doing some change to Yarn source code. I have changed hadoop.root.logger=DEBUG,console to DEBUG in ${…

hadoop logging log4j yarn resourcemanager

SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS] While getting spark context

Installation Info :- Hadoop version :- 2.6.5 Spark Version :- 2.1.0 And Kerberos I am trying to get the spark context in …

authentication apache-spark yarn kerberos hadoop2

Slurm: What is the difference for code executing under salloc vs srun

I'm using a cluster managed by slurm to run some yarn/hadoop benchmarks. To do this I am starting the …

hadoop yarn slurm

Can sparklyr be used with spark deployed on yarn-managed hadoop cluster?

Is the sparklyr R package able to connect to YARN-managed hadoop clusters? This doesn't seem to be documented in the …

r apache-spark yarn sparkapi sparklyr

Know the disk space of data nodes in hadoop?

Is there a way or any command using which I can come to know the disk space of each datanode …

hadoop hdfs yarn hadoop2 webhdfs

Apache Spark YARN mode startup takes too long (10+ secs)

I’m running a spark application with YARN-client or YARN-cluster mode. But it seems to take too long to startup. …

hadoop apache-spark yarn