Spark workers stopped after driver commanded a shutdown

apache-spark apache-spark-standalone
pooshan Singh picture pooshan Singh · Dec 6, 2016 · Viewed 7.5k times · Source

Basically, Master node also perform as a one of the slave. Once slave on master completed it called the SparkContext to stop and hence this command propagate to all the slaves which stop the execution in mid of the processing.

Error log in one of the worker:

INFO SparkHadoopMapRedUtil: attempt_201612061001_0008_m_000005_18112: Committed

INFO Executor: Finished task 5.0 in stage 8.0 (TID 18112). 2536 bytes result sent to driver

INFO CoarseGrainedExecutorBackend: Driver commanded a shutdown

ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERMtdown

Answer

user500377 picture user500377 · Apr 19, 2017

Check your resource manager user interface, in case you see any executor failed - it details about memory error. However if executor has not failed but still driver called for shut down - usually this is due to driver memory, please try to increase driver memory. Let me know how it goes.

Related questions

What is the relationship between workers, worker instances, and executors?

In Spark Standalone mode, there are master and worker nodes. Here are few questions: Does 2 worker instance mean one worker node with 2 worker processes? Does every worker instance hold an executor for specific application (which manages storage, task) or one …

Read More
Apache Spark: Differences between client and cluster deploy modes

TL;DR: In a Spark Standalone cluster, what are the differences between client and cluster deploy modes? How do I set which mode my application is going to run on? We have a Spark Standalone cluster with three machines, all …

Read More
Which cluster type should I choose for Spark?

I am new to Apache Spark, and I just learned that Spark supports three types of cluster: Standalone - meaning Spark will manage its own cluster YARN - using Hadoop's YARN resource manager Mesos - Apache's dedicated resource manager project …

Read More