How to check status of Spark (Standalone) services on cloudera-quickstart-vm?

Question

I am trying to get the status of the services namely spark-master and spark-slaves running on Spark (standalone) service running on my local vm

However running sudo service spark-master status is not working.

Can anybody provide some hints on how to check the status of Spark services?

Answer 1

I use jps -lm as the tool to get status of any JVMs on a box, Spark's ones including. Consult jps documentation for more details beside -lm command-line options.

If you however want to filter out the JVM processes that really belong to Spark you should pipe it and use OS-specific tools like grep.

➜  spark git:(master) ✗ jps -lm
999 org.apache.spark.deploy.master.Master --ip japila.local --port 7077 --webui-port 8080
397
669 org.jetbrains.idea.maven.server.RemoteMavenServer
1198 sun.tools.jps.Jps -lm

➜  spark git:(master) ✗ jps -lm | grep -i spark
999 org.apache.spark.deploy.master.Master --ip japila.local --port 7077 --webui-port 8080

You can also check out ./sbin/spark-daemon.sh status, but my limited understanding of the tool doesn't make it a recommended one.

When you start Spark Standalone using scripts under sbin, PIDs are stored in /tmp directory by default. ./sbin/spark-daemon.sh status can read them and do the "boilerplate" for you, i.e. status a PID.

➜  spark git:(master) ✗ jps -lm | grep -i spark
999 org.apache.spark.deploy.master.Master --ip japila.local --port 7077 --webui-port 8080

➜  spark git:(master) ✗ ls /tmp/spark-*.pid
/tmp/spark-jacek-org.apache.spark.deploy.master.Master-1.pid

➜  spark git:(master) ✗ ./sbin/spark-daemon.sh status org.apache.spark.deploy.master.Master 1
org.apache.spark.deploy.master.Master is running.