How to install Apache Toree for Spark Kernel in Jupyter in (ana)conda environment?

Question

I am trying to install Jupyter-support for Spark in a conda environment (which I set up using http://conda.pydata.org/docs/test-drive.html) of the anaconda distribution. I am trying to use apache toree as Jupyter Kernel for this.

Here is what I did after I installed Anaconda:

conda create --name jupyter python=3
source activate jupyter
conda install jupyter
pip install --pre toree
jupyter toree install

Everything worked fine until I reached the last line. There I get

PermissionError: [Errno 13] Permission denied: '/usr/local/share/jupyter'

Which begs the question: Why is it even looking in that directory? Afterall it is supposed to stay in the environment. Thus I exectue

jupyter --paths

and get

config:
    /home/user/.jupyter
    ~/anaconda2/envs/jupyter/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/user/.local/share/jupyter
    ~/anaconda2/envs/jupyter/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /run/user/1000/jupyter

I am not quite sure what is going on and how to proceed to get everything running in (and if possible only in) the conda environment "jupyter".

Answer 1

Jupyter tries to install kernel into systemwide kernel registry by default. You can pass a --user flag and it will use a user kernel dir. More details are available in kernelspec.py. Following command installs toree kernel into the user kernel

jupyter toree install --user