This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author loewis
Recipients Max.Skaller, loewis
Date 2010-10-30.07:40:53
SpamBayes Score 0.0
Marked as misclassified No
Message-id <4CCBCC03.8020200@v.loewis.de>
In-reply-to <1288421190.78.0.406160800778.issue10243@psf.upfronthosting.co.za>
Content
> Python 2.6, for example, does ship a shared library.

That is not really the case. Python 2.6 ships in source form. It
builds with a libpython shared library only if you configure
with --enable-shared.

> In particular it is NOT acceptable for any extension or shared
> library component to expect to find its symbols in the host
> application executable as the Wiki documentation seems to suggest (in
> a section which explains a bit of a workaround for OSX frameworks).

I completely disagree with this evaluation. These statements are not
true in general, although they might be partially true for specific
operating systems and compilers.

> Although the details are platform specific, it is simply
> not safe to permit C++ extension modules unless this step is taken.

It certainly is safe on all platforms where Python does that,
i.e. primarily Linux, Windows, and OS X. Only archaic C++
implementations fail to support applications where main()
was not compiled with a C++ compiler.

> On most major platforms, however, a C++ driver stub is required.

That is not true.

> It is not only possible, it
> is in fact the case for my project, that were the Python interpreter
> to be called, it would from a shared library loaded under program
> control at run time.

Python supports embedding of the interpreter quite well. You can
choose between linking Python as a static (.a) or dynamic library.
If you use a static library, extension modules would still pick
up Python symbols from the executable.

> Such an interpreter cannot be loaded at all if it isn't present in a
> library: it either has to be statically linked into the shared
> library making the call, with some ugly linker switches to make sure
> no symbols are dropped, or it has to be loaded dynamically.

The Python build process will *always* create a library that embedding
applications can link against.

> For a generated program to call Python interpreter, it HAS to be
> available in a shared library, and for any extension modules that
> interpreter loads, they HAVE to get their symbols from that shared
> library, and, if the generated program is itself a Python module,
> then if that module is to be loaded from any C extension, including
> itself or some other extension, it HAS to be linked against libpython
> which HAS to be loaded dynamically by the loaded.

If that's your requirement, make sure you build against a Python
installation that was itself built with --enable-shared.

> The unfortunate downside of this is that it is NOT POSSIBLE to have a
> huge statically linked Python executable which just loads C
> extensions and nothing else happens. If you're loading any C
> extensions dynamically libpython must be loaded dynamically too.

That depends on the operating system. On Linux, it would be possible
to link the entire Python interpreter in your language's shared library,
and extension modules would still find their symbols.

> Just to be clear: I can easily build it the way I want it but this
> will not solve my problem, which is to support clients who wish to
> use my product to generate high performance Python callable modules
> which will just work "out of the box" with existing Python code. In
> particular, replacing some slow modules with optimised ones would be
> more or less entirely transparent .. except that at the moment it
> could only work with Python 2.x since Python 3.x shipments don't seem
> to have any shared libpython included (and I just changed my compiler
> to support Python 3 modules instead of Python 2).

One solution could be to ship libpythonXY.so yourself, along with your
the rest of the binaries that you provide. Or else do as I propose
above: link the Python interpreter statically into your shared library.

I fail to see the bug in this report.
History
Date User Action Args
2010-10-30 07:40:55loewissetrecipients: + loewis, Max.Skaller
2010-10-30 07:40:54loewislinkissue10243 messages
2010-10-30 07:40:53loewiscreate