How to check the Spark version

apache-spark cloudera-cdh
HappyCoding picture HappyCoding · Apr 17, 2015 · Viewed 102.1k times · Source

as titled, how do I know which version of spark has been installed in the CentOS?

The current system has installed cdh5.1.0.

Answer

Shyamendra Solanki picture Shyamendra Solanki · Apr 17, 2015

If you use Spark-Shell, it appears in the banner at the start.

Programatically, SparkContext.version can be used.

Related questions

Spark : how to run spark file from spark shell

I am using CDH 5.2. I am able to use spark-shell to run the commands. How can I run the file(file.spark) which contain spark commands. Is there any way to run/compile the scala programs in CDH 5.2 without sbt? …

Read More
Cannot Read a file from HDFS using Spark

I have installed cloudera CDH 5 by using cloudera manager. I can easily do hadoop fs -ls /input/war-and-peace.txt hadoop fs -cat /input/war-and-peace.txt this above command will print the whole txt file on the console. now I start …

Read More
Class com.hadoop.compression.lzo.LzoCodec not found for Spark on CDH 5?

I have been working on this problem for two days and still have not find the way. Problem: Our Spark installed via newest CDH 5 always complains about the lost of LzoCodec class, even after I install the HADOOP_LZO through …

Read More