Issue42307
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020-11-10 10:49 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Pull Requests | |||
---|---|---|---|
URL | Status | Linked | Edit |
PR 23219 | closed | vstinner, 2020-11-10 11:44 |
Messages (13) | |||
---|---|---|---|
msg380650 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-11-10 10:49 | |
Commands reproduce the issue: cd /path/to/python/source/ ./configure --prefix /opt/pymaster CFLAGS="-O0" make make install make install copies python.o: $ find /opt/pymaster/ -name "*.o" /opt/pymaster/lib/python3.10/config-3.10-x86_64-linux-gnu/python.o This file is useless and must not be installed by make install. The issue was first discovered in Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=1894462 |
|||
msg380652 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-11-10 11:13 | |
python.o is installed by "make libainstall". It is done since 2001 (commit 85515ad9795ffc3b676cbddeeea2b003818a2623). Git history: commit 49fd7fa4431da299196d74087df4a04f99f9c46f Author: Thomas Wouters <thomas@python.org> Date: Fri Apr 21 10:40:58 2006 +0000 Merge p3yk branch with the trunk up to revision 45595. This breaks a fair number of tests, all because of the codecs/_multibytecodecs issue described here (it's not a Py3K issue, just something Py3K discovers): http://mail.python.org/pipermail/python-dev/2006-April/064051.html (...) - $(INSTALL_DATA) Modules/$(MAINOBJ) $(DESTDIR)$(LIBPL)/$(MAINOBJ) + $(INSTALL_DATA) Modules/python.o $(DESTDIR)$(LIBPL)/python.o commit a1a84e7d4f7b493bb6fa5415ce55d3f722221470 Author: Fred Drake <fdrake@acm.org> Date: Tue Mar 6 05:52:16 2001 +0000 Move all knowledge that $(MAINOBJ) is built in the Modules/ directory into Makefile.pre.in; the configure script will only determine the basename of the file. This fixes installation of a Python built using C++, reported by Greg Wilson. - $(INSTALL_DATA) Modules/python.o $(LIBPL)/python.o + $(INSTALL_DATA) Modules/$(MAINOBJ) $(LIBPL)/$(MAINOBJ) commit 85515ad9795ffc3b676cbddeeea2b003818a2623 Author: Neil Schemenauer <nascheme@enme.ucalgary.ca> Date: Wed Jan 24 17:11:43 2001 +0000 Flat makefile based on toplevel Makefile.in and makefiles in build subdirectories. Those other makefiles will go away eventually. + $(INSTALL_DATA) Modules/python.o $(LIBPL)/python.o |
|||
msg380653 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-11-10 11:27 | |
At commit 85515ad9795ffc3b676cbddeeea2b003818a2623, "make install" installs the following files in the config/ directory: $ find /opt/pyold/lib/python2.1/config/ /opt/pyold/lib/python2.1/config/ /opt/pyold/lib/python2.1/config/libpython2.1.a /opt/pyold/lib/python2.1/config/config.c /opt/pyold/lib/python2.1/config/python.o /opt/pyold/lib/python2.1/config/config.c.in /opt/pyold/lib/python2.1/config/Makefile /opt/pyold/lib/python2.1/config/Setup /opt/pyold/lib/python2.1/config/Setup.local /opt/pyold/lib/python2.1/config/Setup.config /opt/pyold/lib/python2.1/config/makesetup /opt/pyold/lib/python2.1/config/install-sh /opt/pyold/lib/python2.1/config/Makefile.pre.in |
|||
msg380655 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-11-10 11:47 | |
python.o is the object file build by the C compiler from Programs/python.c. I don't see why anyone would need such object file. Programs/python.c: --- /* Minimal main program -- everything is loaded from the library */ #include "Python.h" #ifdef MS_WINDOWS int wmain(int argc, wchar_t **argv) { return Py_Main(argc, argv); } #else int main(int argc, char **argv) { return Py_BytesMain(argc, argv); } #endif --- $ objdump -d /opt/pymaster/lib/python3.10/config-3.10-x86_64-linux-gnu/python.o (...) Disassembly of section .text: 0000000000000000 <main>: 0: 55 push %rbp 1: 48 89 e5 mov %rsp,%rbp 4: 48 83 ec 10 sub $0x10,%rsp 8: 89 7d fc mov %edi,-0x4(%rbp) b: 48 89 75 f0 mov %rsi,-0x10(%rbp) f: 48 8b 55 f0 mov -0x10(%rbp),%rdx 13: 8b 45 fc mov -0x4(%rbp),%eax 16: 48 89 d6 mov %rdx,%rsi 19: 89 c7 mov %eax,%edi 1b: e8 00 00 00 00 callq 20 <main+0x20> 20: c9 leaveq 21: c3 retq $ objdump -t /opt/pymaster/lib/python3.10/config-3.10-x86_64-linux-gnu/python.o (...) 0000000000000000 *UND* 0000000000000000 Py_BytesMain |
|||
msg380671 - (view) | Author: Ned Deily (ned.deily) * | Date: 2020-11-10 14:17 | |
I don't know for sure why python.o is included but perhaps the intent was to allow a user to rebuild a static interpreter with an user-supplied extension as is hinted at in https://docs.python.org/dev/extending/extending.html#compilation-and-linkage |
|||
msg380674 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-11-10 14:30 | |
Oh! I managed to build a static Python with the following commands: cd /opt/pymaster/lib/python3.10/config-3.10-x86_64-linux-gnu gcc -static -o ~/python-static python.o -L. $(./python-config.py --libs --embed) I get a static binary: $ ldd ~/python-static not a dynamic executable And it works as expected: $ ~/python-static Python 3.10.0a2+ (heads/master:1f73c320e2, Nov 10 2020, 11:47:55) [GCC 10.2.1 20201016 (Red Hat 10.2.1-6)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> But... does any user really build Python manually after Python is installed? The Python build system couldn't handle that as part of the regular build? Maybe using a new configure --enable-static flag? I'm not a fan of static binaries. For example, the linker emits many warnings about static linking: /usr/bin/ld: ./libpython3.10.a(dynload_shlib.o): in function `_PyImport_FindSharedFuncptr': /home/vstinner/python/master/./Python/dynload_shlib.c:100: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/bin/ld: ./libpython3.10.a(posixmodule.o): in function `os_getgrouplist_impl': /home/vstinner/python/master/./Modules/posixmodule.c:7411: warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/bin/ld: ./libpython3.10.a(posixmodule.o): in function `os_initgroups_impl': /home/vstinner/python/master/./Modules/posixmodule.c:7610: warning: Using 'initgroups' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/bin/ld: ./libpython3.10.a(pwdmodule.o): in function `pwd_getpwall_impl': /home/vstinner/python/master/./Modules/pwdmodule.c:302: warning: Using 'getpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/bin/ld: /home/vstinner/python/master/./Modules/pwdmodule.c:301: warning: Using 'setpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/bin/ld: /home/vstinner/python/master/./Modules/pwdmodule.c:307: warning: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/bin/ld: ./libpython3.10.a(pwdmodule.o): in function `pwd_getpwnam_impl': /home/vstinner/python/master/./Modules/pwdmodule.c:249: warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/bin/ld: ./libpython3.10.a(pwdmodule.o): in function `pwd_getpwuid': /home/vstinner/python/master/./Modules/pwdmodule.c:166: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking |
|||
msg380682 - (view) | Author: Pablo Galindo Salgado (pablogsal) * | Date: 2020-11-10 16:03 | |
> But... does any user really build Python manually after Python is installed? The Python build system couldn't handle that as part of the regular build? Maybe using a new configure --enable-static flag? Embedding? But they should use the .a not any of the .o. Object files should not be packaged IMHO. |
|||
msg380685 - (view) | Author: Ned Deily (ned.deily) * | Date: 2020-11-10 16:08 | |
> Embedding? The example I cited from the docs was for extending Python by building a new interpreter using an installed Python, not embedding. Dunno how often that is done, though. Perhaps on Windows where it was more difficult to build an interpreter from source? Perhaps Nick might have some ideas since he was involved in that section of the docs. |
|||
msg380711 - (view) | Author: Neil Schemenauer (nascheme) * | Date: 2020-11-10 20:42 | |
I think the comments are correct in that it is used to create a new statically linked interpreter that includes a user provided extension module. We could include python.o inside the libpythonXX.a file but then I think it breaks if you are embedding (e.g. linking .a to your own .o that has a main function). It seems probable that no one uses python.o anymore but my preference would be to not remove it unless we are going to completely remove support for statically linking. Nothing is really hurt by having that .o file in the install and the feature still works as originally intended. It would be good to add a comment in the makefile, describing the possible use for that file. |
|||
msg380722 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-11-10 22:30 | |
The Fedora package of Python doesn't build the static libpython for 10 years (patch added at 2010-01-18): https://src.fedoraproject.org/rpms/python3.9/blob/master/f/00111-no-static-lib.patch libpython3.10.a (static) is around 15 MB and libpython3.10.so (dynamic) is around 3 MB. Fedora only installs libpython3.10.so (dynamic). |
|||
msg380724 - (view) | Author: Pablo Galindo Salgado (pablogsal) * | Date: 2020-11-10 22:35 | |
Hummm, I think the gains of having the .o over the .a are very very small as you can almost recreate anything you want with the .o using the .a and a small initialization function unless I am missing something. |
|||
msg386150 - (view) | Author: STINNER Victor (vstinner) * | Date: 2021-02-02 15:24 | |
I created bpo-43103 "Add configure option to disable build libpython.a and don't install python.o". |
|||
msg387144 - (view) | Author: STINNER Victor (vstinner) * | Date: 2021-02-17 10:59 | |
I consider that this issue is now fixed by bpo-43103 with the addition of a new configure --without-static-libpython option. If someone prefers to remove python.o and get it from libpython.a, please open a separated issue. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:37 | admin | set | github: 86473 |
2021-02-17 10:59:40 | vstinner | set | status: open -> closed superseder: Add configure --without-static-libpython to not build libpython3.10.a messages: + msg387144 resolution: duplicate stage: patch review -> resolved |
2021-02-02 15:24:07 | vstinner | set | messages: + msg386150 |
2021-01-27 10:28:02 | hroncok | set | nosy:
+ hroncok |
2020-11-10 22:35:05 | pablogsal | set | messages: + msg380724 |
2020-11-10 22:30:20 | vstinner | set | messages: + msg380722 |
2020-11-10 20:42:52 | nascheme | set | messages: + msg380711 |
2020-11-10 16:08:10 | ned.deily | set | nosy:
+ ncoghlan messages: + msg380685 |
2020-11-10 16:03:10 | pablogsal | set | messages: + msg380682 |
2020-11-10 14:30:03 | vstinner | set | messages: + msg380674 |
2020-11-10 14:17:03 | ned.deily | set | messages: + msg380671 |
2020-11-10 11:47:54 | vstinner | set | nosy:
+ ned.deily, pablogsal messages: + msg380655 |
2020-11-10 11:44:07 | vstinner | set | keywords:
+ patch stage: patch review pull_requests: + pull_request22117 |
2020-11-10 11:27:46 | vstinner | set | messages: + msg380653 |
2020-11-10 11:13:08 | vstinner | set | nosy:
+ nascheme messages: + msg380652 |
2020-11-10 10:49:40 | vstinner | create |