Python can't locate distutils_path on Mac OSX

Question

I've been using virtualenv + pip for python development. I'm not sure what happened, but suddenly whenever I try to run a command-line tool or import libraries, I get this error message:

Traceback (most recent call last):
  File "/Users/kyle/.virtualenvs/fj/bin/pip", line 4, in <module>
    import pkg_resources
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 698, in <module>
    the platform/python version defined at initialization are added.
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 701, in Environment
    search_path = sys.path
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 96, in get_supported_platform
    'Environment', 'WorkingSet', 'ResourceManager',
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 221, in get_build_platform
    if provDarwin:
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/distutils/__init__.py", line 14, in <module>
    exec open(os.path.join(distutils_path, '__init__.py')).read()
IOError: [Errno 2] No such file or directory: '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/__init__.py'

From what I can decipher, Python is trying to find distutils_path in the Mac OSX system version Python, not my virtualenv version like it should be.

Not sure why this suddenly started happening. Maybe a recent OSX update? Another possibility is that my hard drive was about to die, so Apple gave me a new one and ran Migration Assistant. Maybe something didn't transferred across correctly?

Answer 1

I encountered this distutils/__init__.py problem when transitioning to OS X 10.7 Lion (from OS X 10.5 Leopard) and using Migration Assistant. I've already installed Xcode 3.2.6 -- thus resolving the missing install_name_tool problem.

Migration Assistant brought over my previous virtualenvs, but since they were based on Leopard's Python 2.5, I figure I need to recreate each of them with the current system Python 2.7.

easy_install was already in the PATH -- probably because it was bundled with Lion's Python 2.7; it seems unlikely to be the result of Migration Assistant. I used easy_install to install virtualenv.

This problem, it seems to me, doesn't have anything to do with Xcode or lack thereof. It's a peculiar line in a file placed in the new virtual env by the virtualenv command:

  File "/path/to/my/virtualenv/lib/python2.7/distutils/__init__.py", line 16, in 
    exec(open(os.path.join(distutils_path, '__init__.py')).read())
IOError: [Errno 2] No such file or directory: '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/__init__.py'

The issue is that, in the Python 2.7 install bundled with Lion, the library doesn't come with .py source files. That directory contains .pyc and .pyo files, but no .py files. virtualenv doesn't seem to expect that.

My workaround is to download Python 2.7 source:
http://python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2

and unpack distutils/__init__.py into the expected place:
sudo tar xvjf ~/Downloads/Python-2.7.2.tar.bz2 --strip-components=2 -C /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7 Python-2.7.2/Lib/distutils/__init__.py

That permits virtualenv to complete successfully, and the resulting Python interpreter seems to run.

Given that the Python 2.7 library bundled with Lion is installed without source, it might seem useful to change virtualenv to try for either distutils/__init__.py or distutils/__init__.pyc ?