Python: select one of multiple installed module versions

Question

On my system, I have several modules installed multiple times. To give an example, numpy 1.6.1 is installed in the standard path at /usr/lib/python2.7/dist-packages, and I have an updated version of numpy 1.8.0 installed at /local/python/lib/python2.7/site-packages/.

The reason I cannot simply remove the old version is that I do not have permissions to change anything on my work computer. I however need to use the new numpy version.

I have added /local/python/lib/python2.7/site-packages/ to my PYTHONPATH. Unfortunately, this does not help, since /usr/lib/python2.7/dist-packages is inserted into the path first and therefore, numpy 1.6.1 will be loaded. Here's an example:

>>> import os
>>> print os.environ['PYTHONPATH']
/local/python/lib/python2.7/site-packages
>>> import pprint
>>> import sys
>>> pprint.pprint(sys.path)
['',
 '/local/python/lib/python2.7/site-packages/matplotlib-1.3.1-py2.7-linux-x86_64.egg',
 '/local/python/lib/python2.7/site-packages/pyparsing-2.0.1-py2.7.egg',
 '~/.local/lib/python2.7/site-packages/setuptools-3.4.4-py2.7.egg',
 '~/.local/lib/python2.7/site-packages/mpldatacursor-0.5_dev-py2.7.egg',
 '/usr/lib/python2.7/dist-packages',
 '/local/python/lib/python2.7/site-packages',
 '/usr/lib/python2.7',
 ...,
 '~/.local/lib/python2.7/dist-packages', 
 ...]

So, it seems that the import order is

1. current directory
2. eggs from PYTHONPATH
3. eggs from local module path (~/.local/lib/python2.7/site-packages/*.egg)
4. system-wide module path (~/usr/lib/python2.7/dist-packages/)
5. directories from PYTHONPATH
6. intermediate paths (omitted for brevity)
7. userbase directory (~/.local/lib/python2.7/site-packages/)

My problem is that I would need to put item 5. before items 3. and 4. for my code to work properly. Right now, if I import a module that was compiled against numpy 1.8.0 from the /local/* directory, and this module imports numpy, it will still take numpy from the /usr/* directory and fail.

I have circumvented this problem by placing something like this in my scripts:

import sys
sys.path.insert(0, '/local/python/lib/python2.7/site-packages/')

Thereby I can force Python to use the right import order, but of course this is not a solution, since I would have to do this in every single script.

Answer 1

Besides the suggestions already given in the comment section, have you thought about using virtualenv? This would give you fine-grained control over every module that you want to use. If you're not familiar with virtualenv you'll want to read the documentation to get a feel for how it works.

Purely for example, you could install and set it up, like so (virtualenv-1.11.6 looks to be the most recent version currently):

$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz
$ tar xvfz virtualenv-1.11.6.tar.gz
$ cd virtualenv-1.11.6
$ python virtualenv.py ../numpyvenv
$ cd ../numpyvenv
$ source ./bin/activate
(numpyvenv) $ pip install numpy
# downloads, compiles, and installs numpy into the virtual environemnt
(numpyvenv) $ python
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.version.version
'1.9.1'
>>> quit()
(numpyvenv) $ deactivate
$ # the virtual environment has been deactivated

Above, we created a virtual environment named "numpyvenv", activated the environment, installed numpy, printed the numpy version (to show it works), quit python, and deactivated the environment. Next time you activate the environment, numpy will be there along with whatever other modules you install. You may run into hiccups while trying this, but it should get you started.