Running Spark on Linux : $JAVA_HOME not set error

linux apache-spark java-home ubuntu-16.04
Marc Zaharescu picture Marc Zaharescu · Aug 3, 2016 · Viewed 7.4k times · Source

I am trying to configure spark-2.0.0-bin-hadoop2.7 on Ubuntu 16.04.1 LTS. I have set 

export JAVA_HOME=/home/marc/jdk1.8.0_101
export SCALA_HOME=/home/marc/scala-2.11.8
export SPARK_HOME=/home/marc/spark-2.0.0-bin-hadoop2.7
export PATH=$PATH:$SCALA_HOME/bin:$JAVA_HOME/bin

at the end of .bashrc and also included in the start-all.sh file from spark/sbin folder

when I type echo $JAVA_HOME it gives me the correct path as /home/marc/jdk1.8.0_101

But when I call sbin/start-all.sh

It gives me the following error

localhost: failed to launch org.apache.spark.deploy.worker.Worker: localhost: JAVA_HOME is not set

I tried to follow similar topics, but I couldn't find a solution to the problem. Any help would be much appreciated.

Answer

Haoran Yang picture Haoran Yang · Dec 16, 2018

You need to modify the file named 'spark-config.sh' in the 'sbin'. Add your JAVA_HOME in this file, then everything will be OK.

Related questions

How do I find all files containing specific text on Linux?

I'm trying to find a way to scan my entire Linux system for all files containing a specific string of text. Just to clarify, I'm looking for text within the file, not in the file name. When I was looking …

Read More
How to change permissions for a folder and its subfolders/files in one step?

I would like to change permissions of a folder and all its sub folders and files in one step (command) in Linux. I have already tried the below command but it works only for the mentioned folder: chmod 775 /opt/lampp/…

Read More
How can I recursively find all files in current and subfolders based on wildcard matching?

How can I recursively find all files in current and subfolders based on wildcard matching?

Read More