classification
Title: Some C-API symbols (e.g. Py_FrozenMain) are not always exported
Type: Stage: patch review
Components: C API Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: kulikjak, miss-islington, petr.viktorin, shihai1991, vstinner
Priority: normal Keywords: patch

Created on 2021-05-14 16:58 by vstinner, last changed 2021-12-01 09:20 by shihai1991.

Files
File name Uploaded Description Edit
reproduce.tar.gz vstinner, 2021-05-14 16:58
Pull Requests
URL Status Linked Edit
PR 26130 closed vstinner, 2021-05-14 17:05
PR 29148 closed miss-islington, 2021-10-22 08:12
PR 29876 open shihai1991, 2021-12-01 09:20
Messages (7)
msg393675 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-05-14 16:58
When Python is built without --enable-shared, the "Py_FrozenMain" is not exported.

When Python is built with --enable-shared, the "Py_FrozenMain" is exported as expected.

I can reproduce the issue with attached reproduce.tar.gz example:

* build.sh exports "func1" symbol
* build_ar.sh doesn't export the "func1" symbol

The difference is that build.sh links directly two object files (.o) into a binary, whereas build_ar.sh creates a static library (.a) and then link the static library into a binary.

Python is always built with a static library (libpythonVERSION.a) which causes the issue.

A solution is to pass -Wl,--whole-archive option to the linker to export *all* symbols exported by all object files, and not only export symbols of the object files which are used.

I'm not sure why, but "Py_FrozenMain" is the *only* impacted symbol of the whole C API.

This issue was discovered by Petr Viktorin who works on the stable ABI:
https://mail.python.org/archives/list/capi-sig@python.org/thread/5QLI3NUP3OSGLCCIBAQOTX4GEJQBWJ6F/
msg393676 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-05-14 17:00
The "python" binary exports 1610 symbols. With the attached fix, it exports 1611 symbols (+1): "Py_FrozenMain" is also exported :-)
msg393693 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-05-14 21:14
The symbol is not exported on Windows neither. My PR 26126 ("Test Py_FrozenMain()") fails on Windows with:

"_testembed.obj : error LNK2001: unresolved external symbol __imp_Py_FrozenMain"

("__imp_" prefix is added by Windows DLL loader.)
msg404596 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-10-21 13:04
> I'm not sure why, but "Py_FrozenMain" is the *only* impacted symbol of the whole C API.

Apparently, on some platforms `PyModule_Create2` and `PyModule_FromDefAndSpec2` don't appear either.

Should I rename the issue to cover these as well?
msg404723 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-10-22 08:12
New changeset 276468dddb46c54980c782c09cdb53bd90755752 by Petr Viktorin in branch 'main':
bpo-43795: Add a test for Stable ABI symbol availability using ctypes (GH-26354)
https://github.com/python/cpython/commit/276468dddb46c54980c782c09cdb53bd90755752
msg404748 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-10-22 10:35
Repurposing this bug to track all symbols that don't appear in some configurations.

- `Py_FrozenMain` should be added to stable ABI when this is fixed. It was left out of the 3.10 stable ABI for only this reason. See discussion here: https://mail.python.org/archives/list/capi-sig@python.org/thread/5QLI3NUP3OSGLCCIBAQOTX4GEJQBWJ6F/

- `PyModule_Create2` and `PyModule_FromDefAndSpec2` are "temporarily" excluded from tests in `Tools/scripts/stable_abi.py`'s `gen_ctypes_test`, so that all the other symbols can get regression testing. When this issue is fixed, that special case should be removed.
msg406980 - (view) Author: Jakub Kulik (kulikjak) * Date: 2021-11-25 08:55
On Solaris (and most likely several other platforms), `PyThread_get_thread_native_id` is also not available.
History
Date User Action Args
2021-12-01 09:20:15shihai1991setnosy: + shihai1991
pull_requests: + pull_request28101
2021-11-25 08:55:51kulikjaksetnosy: + kulikjak
messages: + msg406980
2021-10-22 10:35:46petr.viktorinsetmessages: + msg404748
title: "Py_FrozenMain" symbol is not exported -> Some C-API symbols (e.g. Py_FrozenMain) are not always exported
2021-10-22 08:12:16miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27425
2021-10-22 08:12:11petr.viktorinsetmessages: + msg404723
2021-10-21 13:04:00petr.viktorinsetnosy: + petr.viktorin
messages: + msg404596
2021-05-14 21:14:48vstinnersetmessages: + msg393693
2021-05-14 17:05:10vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request24769
2021-05-14 17:00:16vstinnersetmessages: + msg393676
2021-05-14 16:58:44vstinnercreate