Undefined Symbol in C++ When Loading a Python Shared Library

Question

I have been trying to get a project of mine to run but I have run into trouble. After much debugging I have narrowed down the problem but have no idea how to proceed.

Some background, I am using a python script inside C++ code. This is somewhat documented on Python, and I managed to get it running very well in my basic executable. #include and a -lpython2.6 and everything was grand.

However, difficulty has arisen when running this python script from a shared library(.so). This shared library is "loaded" as a "module" by a simulation system (OpenRAVE). The system interacts with this module using a virtual method for "modules" called SendCommand. The module then starts a boost::thread, giving python its own thread, and returns to the simulation system. However, when python begins importing its modules and thus loading its dynamic libraries it fails, I assume due to the following error:

 ImportError: /usr/lib/python2.6/dist-packages/numpy/core/multiarray.so: undefined symbol: _Py_ZeroStruct 

I have run ldd on my executable and the shared library, there doesn't some to be a difference. I have also run nm -D on the file above, the _Py_ZeroStruct is indeed undefined. If you guys would like print outs of the commands I would be glad to supply them. Any advice would be greatly appreciated, thank you.

Here is the full python error:

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/numpy/__init__.py", line 130, in 
    import add_newdocs
  File "/usr/lib/python2.6/dist-packages/numpy/add_newdocs.py", line 9, in 
    from lib import add_newdoc
  File "/usr/lib/python2.6/dist-packages/numpy/lib/__init__.py", line 4, in 
    from type_check import *
  File "/usr/lib/python2.6/dist-packages/numpy/lib/type_check.py", line 8, in 
    import numpy.core.numeric as _nx
  File "/usr/lib/python2.6/dist-packages/numpy/core/__init__.py", line 5, in 
    import multiarray
ImportError: /usr/lib/python2.6/dist-packages/numpy/core/multiarray.so: undefined symbol: _Py_ZeroStruct
Traceback (most recent call last):
  File "/home/constantin/workspace/OpenRAVE/src/grasp_behavior_2.py", line 3, in 
    from openravepy import *
  File "/home/constantin/workspace/rospackages/openrave/lib/python2.6/site-packages/openravepy/__init__.py", line 35, in 
    openravepy_currentversion = loadlatest()
  File "/home/constantin/workspace/rospackages/openrave/lib/python2.6/site-packages/openravepy/__init__.py", line 16, in loadlatest
    return _loadversion('_openravepy_')
  File "/home/constantin/workspace/rospackages/openrave/lib/python2.6/site-packages/openravepy/__init__.py", line 19, in _loadversion
    mainpackage = __import__("openravepy", globals(), locals(), [targetname])
  File "/home/constantin/workspace/rospackages/openrave/lib/python2.6/site-packages/openravepy/_openravepy_/__init__.py", line 29, in 
    from openravepy_int import *
ImportError: numpy.core.multiarray failed to import

Answer 1

I experienced the same problem with my application and solved it without linking python to the executable.

The setup is as follows:

Executable --links--> library --dynamically-loads--> plugin --loads--> python interpreter

The solution to avoid the ImportErrors was to change the parameters of dlopen, with which the plugin was loaded to RTLD_GLOBAL.

dlopen("plugin.so", RTLD_NOW | RTLD_GLOBAL)

This makes the symbols available to other things loaded afterwards, i.e. other plugins or the python interpreter.

It can, however, happen that symbol clashes occur, because a plugin later exports the same symbols.