Installing anaconda with pyenv, unable to configure virtual environment
I am using pyenv to manage python installations and virtual environments - and I would like anaconda to be one such installation, and to be able to create virtual environments using anaconda python. Using pyenv install anaconda3-2019.03 successfully installs and I can activate the version with pyenv global anaconda3-2019.03:
SamLee-PC:~ max$ pyenv versions
system
2.7.10
2.7.10/envs/flask_tutorial
* 3.7.4 (set by /Users/max/.pyenv/version)
3.7.4/envs/learning_python
3.7.4/envs/microblog
3.7.4/envs/stocktool
3.7.4/envs/test1
anaconda3-2019.03
anaconda3-2019.03/envs/datsci
datsci
flask_tutorial
learning_python
microblog
stocktool
test1
SamLee-PC:~ max$ pyenv which python
/Users/max/.pyenv/versions/3.7.4/bin/python
SamLee-PC:~ max$ pyenv global anaconda3-2019.03
(anaconda3-2019.03) SamLee-PC:~ max$ pyenv which python
/Users/max/.pyenv/versions/anaconda3-2019.03/bin/python
(anaconda3-2019.03) SamLee-PC:~ max$ pyenv which conda
/Users/max/.pyenv/versions/anaconda3-2019.03/bin/conda
(I do not know why the python version is added to the command prompt only with the anaconda installation)
When I creat a new virtual environment (pyenv virtualenv anaconda3-2019.03 datsci), it appears to work as intended:
SamLee-PC:~ max$ pyenv which python
/Users/max/.pyenv/versions/3.7.4/bin/python
SamLee-PC:~ max$ cd code/linkedin/datsci
(datsci) SamLee-PC:datsci max$ pyenv which python
/Users/max/.pyenv/versions/datsci/bin/python
(datsci) SamLee-PC:datsci max$
This is the same relative location returned returned by pyenv which python in my other virtual environments, which all work properly.
The problem is that the python version that gets run in the virtual environment doesn't have access to all of anaconda's packages:
(datsci) SamLee-PC:datsci max$ anaconda-navigator
pyenv: anaconda-navigator: command not found
The `anaconda-navigator' command exists in these Python versions:
anaconda3-2019.03
(datsci) SamLee-PC:datsci max$
though it does if I activate the python installation manually:
SamLee-PC:~ max$ pyenv global anaconda3-2019.03
(anaconda3-2019.03) SamLee-PC:~ max$ anaconda-navigator
WARNING: The conda.compat module is deprecated and will be removed in a future release.
/Users/max/.pyenv/versions/anaconda3-2019.03/lib/python3.7/site-packages/anaconda_navigator/api/conda_api.py:1364: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
data = yaml.load(f)
2019-09-21 14:03:38,666 - ERROR download_api._download:234
Invalid url https://www.anaconda.com/wp-content/uploads/2017/05/Webinar20-20Three20Ways20to20Move20your20Data20Science20Projects20to20Production.png
#anaconda navigator GUI opens
(oddly, the python version that datsci points to does seem to recognize conda:
(datsci) SamLee-PC:datsci max$ pyenv which python
/Users/max/.pyenv/versions/datsci/bin/python
(datsci) SamLee-PC:datsci max$ conda
usage: conda [-h] [-V] command ...
conda is a tool for managing and deploying applications, environments and packages.
...
The problem is that, as I understand it, activating the python version this way whenever I want anaconda means that I only have access to one version of anaconda python - defeating the purpose of using it in a virtual environment.
One more piece of the puzzle - I noticed that for my other virtual environments, ~/.pyenv/versions/name-of-virtualenv contains a file pyvenv.cfg, which contains e.g. the following:
1 home = /Users/max/.pyenv/versions/3.7.4/bin
2 include-system-site-packages = false
3 version = 3.7.4
There was no such file in ~/.pyenv/versions/datsci, so I added one:
1 home = /Users/max/.pyenv/versions/anaconda3-2019.03/bin
2 include-system-site-packages = false
3 version = anaconda3-2019.03
to no avail.
FWIW the contents of that directory are:
(anaconda3-2019.03) SamLee-PC:datsci max$ pwd
/Users/max/.pyenv/versions/datsci
(anaconda3-2019.03) SamLee-PC:datsci max$ ls
bin conda-meta include lib pyvenv.cfg share ssl
(anaconda3-2019.03) SamLee-PC:datsci max$
I wonder if I have the wrong contents in the pyvenv.cfg file for datsci...but I am at a loss.
I really like the way pyenv works, and I would like to continue using it. Is there a way to configure anaconda to behave properly under pyenv?
Update:
I uninstalled my pyenv-managed anaconda and did a fresh install as per Simba's recommendation. Now by default pyenv determines the active python installation, and I can activate conda with conda activate base:
Maxs-MacBook-Air:~ max$ which python
/Users/max/.pyenv/shims/python
Maxs-MacBook-Air:~ max$ conda activate base
(base) Maxs-MacBook-Air:~ max$ which python
/Users/max/anaconda3/bin/python
(base) Maxs-MacBook-Air:~ max$
However after creating a new conda environment with conda create --name datsci, the new conda environment does not seem to activate properly:
Maxs-MacBook-Air:~ max$ conda activate datsci
(datsci) Maxs-MacBook-Air:~ max$ which python
/Users/max/.pyenv/shims/python
What am I missing?
Here are the contents of my .bash_profile:
1 export PATH="/Users/max/.pyenv/bin:$PATH"
2 eval "$(pyenv init -)"
3 eval "$(pyenv virtualenv-init -)"
4
5 # >>> conda initialize >>>
6 # !! Contents within this block are managed by 'conda init' !!
7 __conda_setup="$('/Users/max/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
8 if [ $? -eq 0 ]; then
9 eval "$__conda_setup"
10 else
11 if [ -f "/Users/max/anaconda3/etc/profile.d/conda.sh" ]; then
12 . "/Users/max/anaconda3/etc/profile.d/conda.sh"
13 else
14 export PATH="/Users/max/anaconda3/bin:$PATH"
15 fi
16 fi
17 unset __conda_setup
18 # <<< conda initialize <<<
Is it important that I duplicate the if statement found in Simba's .bash_profile?
Update 2:
Problem solved, I had to install python in the new environment.
Answer
Personal recommendation: Don't use pyenv to install Anaconda or Miniconda.
Both pyenv and conda are able to manage different python environments. The anaconda installed by pyenv should only serves as a Python interpreter. Python environment creation from anaconda installed by pyenv is still handled by pyenv virtualenv but not conda env create.
I've been using these two tools together. It turns out the best solution is to install conda, pyenv separately, and manage their virtual environments separately as well.
- alway initialize
pyenv - only expose command
condabut don't activate any environment
Detail
Install pyenv.
Install Anaconda or Miniconda normally, NOT by pyenv install.
Make sure the pyenv and conda commands are available in an interactive shell.
Initialize pyenv by putting following content into shell init file (.bashrc for Bash, .zshrc for ZSH).
# Put the content into ~/.bashrc or ~/.bash_profile for Bash,
# .zshrc for ZSH
# you may need to add dir of command `pyenv` into PATH,
# if command pyenv is not available yet
if command -v pyenv &>/dev/null; then
eval "$(pyenv init -)"
fi
if command -v pyenv-virtualenv &>/dev/null; then
eval "$(pyenv virtualenv-init -)"
fi
Expose command conda but don't activate any environment, even the base environment. Execute the following commands in your shell.
# Run the content in the shell
# init conda, the following command write scripts into your shell init file automatically
conda init
# disable init of env "base"
conda config --set auto_activate_base false
Note: After this setup, the default python is the one set by pyenv global. Use pyenv and conda to manage environments separately.
Examples of managing virtual environments.
# virtual environments from pyenv
pyenv install 3.6.9
pyenv virtualenv 3.6.9 new-env
pyenv activate new-env
pyenv deactive
# You can also use `pyenv local`
# virtual environments from conda
conda env create new-env python=3.6
conda env list
conda activate new-env
conda deactivate
Default env location for pyenv is ~/.pyenv/versions.
Default env location for conda, check output from conda info.