I would like to keep everything contained within the virtualenv. Is this possible with OpenCV? I'm fine with building from scratch, do I just need to setup the virtualenv first then use special compile flags to tell it where to install to?
I found the solution was that I had to copy over cv2.so and cv.py to the directory running the virtualenv, then pip install numpy. To do this on Ubuntu 12.04 I used.
virtualenv virtopencv
cd virtopencv
cp /usr/local/lib/python2.7/dist-packages/cv* ./lib/python2.7/site-packages/
./bin/pip install numpy
source bin/activate
python
import cv