classification
Title: Embedding into a shared library fails again
Type: behavior Stage:
Components: Documentation Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, doko, pitrou, rinatous, sandro.tosi
Priority: normal Keywords:

Created on 2013-10-03 16:36 by rinatous, last changed 2013-10-04 20:40 by pitrou.

Messages (4)
msg198900 - (view) Author: Rinat (rinatous) Date: 2013-10-03 16:36
I have same error as here described http://bugs.python.org/issue4434

I made everythings according this article http://docs.python.org/2/extending/embedding.html#compiling-and-linking-under-unix-like-systems and more but when i try to call interpriter from C++ it fails with errors

python_support::pre_process_payment() failed
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<hidden-path>/payments/__init__.py", line 1, in <module>
    from .factory import *
  File "<hidden-path>/payments/factory.py", line 7, in <module>
    import psycopg2
  File "/usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py", line 50, in <module>
    from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: /usr/local/lib/python2.7/dist-packages/psycopg2/_psycopg.so: undefined symbol: PyExc_SystemError
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 66, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python2.7/dist-packages/apport/__init__.py", line 1, in <module>
    from apport.report import Report
  File "/usr/lib/python2.7/dist-packages/apport/report.py", line 16, in <module>
    from xml.parsers.expat import ExpatError
  File "/usr/lib/python2.7/xml/parsers/expat.py", line 4, in <module>
    from pyexpat import *
ImportError: /usr/lib/python2.7/lib-dynload/pyexpat.so: undefined symbol: _Py_ZeroStruct

Original exception was:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<hidden-path>/payments/__init__.py", line 1, in <module>
    from .factory import *
  File "<hidden-path>/payments/factory.py", line 7, in <module>
    import psycopg2
  File "/usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py", line 50, in <module>
    from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: /usr/local/lib/python2.7/dist-packages/psycopg2/_psycopg.so: undefined symbol: PyExc_SystemError


Environment
3.2.0-54-generic-pae #82-Ubuntu (12.04)
g++ (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
Boost.Python 1.48
Python 2.7.3 (default, Sep 26 2013, 20:08:41) 
[GCC 4.6.3] on linux2

No custom builds. Every package from ubuntu repository. Psycopg2 installed by pip

Project is big so i can't to provide code
msg198945 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-04 12:37
> No custom builds. Every package from ubuntu repository. Psycopg2 installed by pip

You may have to rebuild packages using the custom Python, not the system Python, since the linker options have to be different (especially the "shared" part: Ubuntu's Python is a static build, not a shared library build).

Or perhaps you can dlopen the CPython shared library with the RTLD_GLOBAL flag. I don't know.

I don't know if this is something that we can alleviate in Python itself.
msg198961 - (view) Author: Rinat (rinatous) Date: 2013-10-04 19:13
> Or perhaps you can dlopen the CPython shared library with the RTLD_GLOBAL flag. I don't know.

Yes, it works. I made it is my shared library before boost block like this

{
  void* handle = dlopen("libpython2.7.so", RTLD_LAZY | RTLD_GLOBAL);
  // boost wrapper for python call
  dlclose(handle);
}


I think if nothing helps, i'll put this code into __attribute__((ctor|dtor)) functions

Thanks a lot
msg198962 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-04 20:40
Ok, making it a documentation issue in case it deserves clarification in the docs.
History
Date User Action Args
2013-10-04 20:40:02pitrousetnosy: + docs@python
messages: + msg198962

assignee: docs@python
components: + Documentation, - Library (Lib)
2013-10-04 19:13:51rinatoussetmessages: + msg198961
2013-10-04 12:37:17pitrousetnosy: + sandro.tosi, doko, pitrou

messages: + msg198945
versions: + Python 3.3, Python 3.4
2013-10-03 16:36:53rinatouscreate