msg382841 - (view) |
Author: Tom Birch (froody) |
Date: 2020-12-10 20:21 |
After https://github.com/python/cpython/pull/12946, there exists an issue on MacOS due to the two-level namespace for symbol resolution. If a C extension links against libpython.dylib, all symbols dependencies from the Python C API will be bound to libpython. When the C extension is loaded into a process launched by running the `python` binary, there will be two active copies of the python runtime. This can lead to crashes if objects from one runtime are used by the other.
https://developer.apple.com/library/archive/documentation/Porting/Conceptual/PortingUnix/compiling/compiling.html#//apple_ref/doc/uid/TP40002850-BCIHJBBF
See issue/test case here: https://github.com/PDAL/python/pull/76
|
msg382860 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2020-12-11 10:22 |
Does this affect unix-style builds with --enable-shared or framework builds?
For the latter the python executable is linked to the framework, and lib python.dylib is an alias for the framework.
A related known problem is that linking an extension to a different interpreter than it was loaded into. That is, link to /opt/lib/libpython.dylib, and use with a framework build (of v.v.). That's one reason why linking extensions to libpython is not commonly done.
It might be interesting to experiment with "-bundle_loader ..." in the link flags instead of linking to libpython, this checks symbols not found in one of the linked to libraries against the bundle_loader ("like it was one of the dynamic libraries the bundle was linked with"). I don't know if this is a recursive check that will also look at libraries linked to by the bundle loader.
|
msg382878 - (view) |
Author: Tom Birch (froody) |
Date: 2020-12-11 19:37 |
> Does this affect unix-style builds with --enable-shared or framework builds?
I believe the answer is no, since in both those cases the `python` executable doesn't contain definitions for any of the libpython symbols. In my testing I was using a python binary from anaconda, where the `python` executable defines all the symbols found in libpython.
|
msg382883 - (view) |
Author: Larry Hastings (larry) * |
Date: 2020-12-11 20:00 |
Stop adding me to this issue.
|
msg382885 - (view) |
Author: Larry Hastings (larry) * |
Date: 2020-12-11 20:03 |
Sorry, somehow bpo decided I added two people to this issue? Weird. Anyway I have removed them (Ned and "froody").
|
msg382907 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2020-12-12 10:18 |
I've removed two spammy comments and adjusted the categories (the latter explains Larry's annoyance, someone selected all categories which automatically added him to the nosy list).
|
msg382908 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2020-12-12 10:26 |
Linking extensions to libpython with a static build of python will cause problems (as you described in your initial message).
The not very satisfactory response is "don't do that then". Note that distutils won't link against libpython by default.
Longer term I want to look into "-bundle_loader ..." because that might allow us to drop "-undefined dynamic_lookup" from the linker flags and hence give a nicer development experience (build-time error instead of runtime error when symbols cannot be found).
|
msg386226 - (view) |
Author: Steve Dower (steve.dower) * |
Date: 2021-02-03 18:03 |
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.
If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools
|
msg386458 - (view) |
Author: Tom Birch (froody) |
Date: 2021-02-03 21:42 |
Steve Dower: this issue is independent of distutils, reopening
|
|
Date |
User |
Action |
Args |
2022-04-11 14:59:39 | admin | set | github: 86782 |
2021-02-03 21:44:00 | froody | set | components:
+ Extension Modules, C API |
2021-02-03 21:42:22 | froody | set | status: closed -> open
messages:
+ msg386458 components:
- Distutils |
2021-02-03 18:03:15 | steve.dower | set | status: open -> closed
nosy:
+ steve.dower messages:
+ msg386226
resolution: out of date stage: resolved |
2020-12-12 10:26:03 | ronaldoussoren | set | messages:
+ msg382908 |
2020-12-12 10:18:51 | ronaldoussoren | set | messages:
+ msg382907 |
2020-12-12 10:17:37 | ronaldoussoren | set | nosy:
+ froody, ned.deily, dstufft, eric.araujo, - asvetlov, yselivanov, Alex.Willmer, zhtw1234 components:
+ Distutils, - 2to3 (2.x to 3.x conversion tool), ctypes, Cross-Build, asyncio, Argument Clinic, C API versions:
- Python 3.6, Python 3.7 |
2020-12-12 10:16:45 | ronaldoussoren | set | messages:
- msg382879 |
2020-12-12 10:16:24 | ronaldoussoren | set | messages:
- msg382880 |
2020-12-11 20:03:52 | larry | set | nosy:
- larry
|
2020-12-11 20:03:44 | larry | set | nosy:
+ larry, - ned.deily, froody messages:
+ msg382885
|
2020-12-11 20:02:09 | larry | set | nosy:
- larry
|
2020-12-11 20:00:31 | larry | set | nosy:
ronaldoussoren, larry, ned.deily, asvetlov, yselivanov, Alex.Willmer, zhtw1234, froody messages:
+ msg382883 |
2020-12-11 19:59:09 | zhtw1234 | set | files:
- IMAG0629_1.jpg |
2020-12-11 19:58:11 | zhtw1234 | set | nosy:
+ larry messages:
+ msg382880
|
2020-12-11 19:56:37 | larry | set | nosy:
- larry
|
2020-12-11 19:54:24 | zhtw1234 | set | files:
+ IMAG0629_1.jpg versions:
+ Python 3.6, Python 3.7, Python 3.10 nosy:
+ zhtw1234, yselivanov, Alex.Willmer, asvetlov, larry
messages:
+ msg382879
components:
+ Build, 2to3 (2.x to 3.x conversion tool), ctypes, Cross-Build, asyncio, Argument Clinic |
2020-12-11 19:37:12 | froody | set | messages:
+ msg382878 |
2020-12-11 10:22:08 | ronaldoussoren | set | priority: normal -> low
messages:
+ msg382860 |
2020-12-10 20:21:24 | froody | create | |