How to check Spark Version

apache-spark hadoop cloudera
Ironman picture Ironman · Jul 26, 2016 · Viewed 111.3k times · Source

I want to check the spark version in cdh 5.7.0. I have searched on the internet but not able to understand. Please help.

Answer

BruceWayne picture BruceWayne · Jul 26, 2016

Addition to @Binary Nerd

If you are using Spark, use the following to get the Spark version:

spark-submit --version

or

Login to the Cloudera Manager and goto Hosts page then run inspect hosts in cluster

Related questions

Where are logs in Spark on YARN?

I'm new to spark. Now I can run spark 0.9.1 on yarn (2.0.0-cdh4.2.1). But there is no log after execution. The following command is used to run a spark example. But logs are not found in the history server as in …

Read More
issue Running Spark Job on Yarn Cluster

I want to run my spark Job in Hadoop YARN cluster mode, and I am using the following command: spark-submit --master yarn-cluster --driver-memory 1g --executor-memory 1g --executor-cores 1 --class com.dc.analysis.jobs.AggregationJob sparkanalitic.jar param1 param2 param3 I am …

Read More
Spark : check your cluster UI to ensure that workers are registered

I have a simple program in Spark: /* SimpleApp.scala */ import org.apache.spark.SparkContext import org.apache.spark.SparkContext._ import org.apache.spark.SparkConf object SimpleApp { def main(args: Array[String]) { val conf = new SparkConf().setMaster("spark://10.250.7.117:7077").setAppName("Simple Application").…

Read More