Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic #54321

Closed
jankratochvil mannequin opened this issue Oct 15, 2010 · 10 comments
Closed

Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic #54321

jankratochvil mannequin opened this issue Oct 15, 2010 · 10 comments
Labels
build The build process and cross-build pending The issue will be closed if no feedback is provided performance Performance or resource usage

Comments

@jankratochvil
Copy link
Mannequin

jankratochvil mannequin commented Oct 15, 2010

BPO 10112
Nosy @loewis, @pitrou, @vstinner, @merwok, @davidmalcolm, @meadori
Files
  • hidden4.patch: Fixed patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2010-10-15.10:16:24.300>
    labels = ['build', 'performance']
    title = 'Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic'
    updated_at = <Date 2019-01-23.15:47:02.674>
    user = 'https://bugs.python.org/jankratochvil'

    bugs.python.org fields:

    activity = <Date 2019-01-23.15:47:02.674>
    actor = 'vstinner'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Build']
    creation = <Date 2010-10-15.10:16:24.300>
    creator = 'jankratochvil'
    dependencies = []
    files = ['24575']
    hgrepos = []
    issue_num = 10112
    keywords = ['patch']
    message_count = 7.0
    messages = ['118756', '151146', '153394', '153459', '153764', '153810', '237136']
    nosy_count = 9.0
    nosy_names = ['loewis', 'pitrou', 'vstinner', 'eric.araujo', 'Arfrever', 'dmalcolm', 'meador.inge', 'jankratochvil', 'tromey']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'resource usage'
    url = 'https://bugs.python.org/issue10112'
    versions = ['Python 3.3']

    @jankratochvil
    Copy link
    Mannequin Author

    jankratochvil mannequin commented Oct 15, 2010

    FSF GDB (and future Fedora GDBs) became 250KB smaller than before. Python recently disabled this GDB build optimization so GDB is 250KB larger again.

    -rwxr-xr-x 1 4524488 gdb-7.1-19.fc13.x86_64/usr/bin/gdb
    -rwxr-xr-x 1 4266728 gdb-7.1-19dynamiclist.fc13.x86_64/usr/bin/gdb
    -rw-r--r-- 1 2364656 gdb-7.1-19.fc13.x86_64.rpm
    -rw-r--r-- 1 2274300 gdb-7.1-19dynamiclist.fc13.x86_64.rpm

    Some Python binaries/libraries could probably also benefit from smaller pageable code image.

    GDB regressed due to /usr/lib*/python*/config/Makefile contains:
    LINKFORSHARED=-Xlinker -export-dynamic

    and GDB thus has to use it even for its own link, effectively disabling its:
    -Wl,--dynamic-list=./proc-service.list

    AFAIK Python already contains the required exports list but it is used only on MS-Windows. It should be utilized even for GNU/Linux builds.

    @jankratochvil jankratochvil mannequin added stdlib Python modules in the Lib dir performance Performance or resource usage labels Oct 15, 2010
    @jankratochvil
    Copy link
    Mannequin Author

    jankratochvil mannequin commented Jan 12, 2012

    Here is the implementation.

    Python/getargs.c was modified for:
    ImportError: /usr/lib64/python2.7/lib-dynload/fcntlmodule.so: undefined symbol: _PyArg_ParseTuple_SizeT
    but I guess that patch part should be different. There is no need to do #ifdef HAVE_DECLSPEC_DLL as PyAPI_FUNC is properly defined even without it. So the declaration+export block should be unified there.

    The Modules/ files were modified for:
    *** WARNING: renaming "_ctypes_test" since importing it failed: dynamic module does not define init function (init_ctypes_test)
    *** WARNING: renaming "_elementtree" since importing it failed: dynamic module does not define init function (init_elementtree)
    *** WARNING: renaming "_hotshot" since importing it failed: dynamic module does not define init function (init_hotshot)
    *** WARNING: renaming "_json" since importing it failed: dynamic module does not define init function (init_json)
    *** WARNING: renaming "ossaudiodev" since importing it failed: dynamic module does not define init function (initossaudiodev)

    without this patch:
    -rw-r--r-- 1 jkratoch jkratoch 5775829 Jan 12 17:42 python-libs-2.7.2-18.fc17.x86_64.rpm
    -r-xr-xr-x 1 jkratoch jkratoch 1738264 Jan 12 17:44 ./clean/python-libs-2.7.2-18.fc17.x86_64/usr/lib64/libpython2.7.so.1.0*
    1226 libpython2.7.so.1.0 .dynsym exports
    +
    -rw-r--r-- 1 jkratoch jkratoch 1986161 Jan 12 17:48 gdb-7.4.50.20120103-8.fc17.x86_64.rpm
    -rwxr-xr-x 1 jkratoch jkratoch 5018800 Jan 12 19:13 clean/gdb-7.4.50.20120103-8.fc17.x86_64/usr/bin/gdb*
    
    with this patch:
    -rw-r--r-- 1 jkratoch jkratoch 5762537 Jan 12 19:04 python-libs-2.7.2-18hidden3.fc17.x86_64.rpm
    -r-xr-xr-x 1 jkratoch jkratoch 1720920 Jan 12 19:06 ./hidden3/python-libs-2.7.2-18hidden3.fc17.x86_64/usr/lib64/libpython2.7.so.1.0*
    1046 libpython2.7.so.1.0 .dynsym exports
    +
    -rw-r--r-- 1 jkratoch jkratoch 1886781 Jan 12 19:11 gdb-7.4.50.20120103-8.fc17.x86_64.rpm
    -rwxr-xr-x 1 jkratoch jkratoch 4732080 Jan 12 19:13 hidden3/gdb-7.4.50.20120103-8.fc17.x86_64/usr/bin/gdb*

    @jankratochvil
    Copy link
    Mannequin Author

    jankratochvil mannequin commented Feb 15, 2012

    What more can be done to get it fixed? I do not mind Python but GDB linking is broken due to it. And while I can workaround it in GDB I am not used for workarounding one Free package in another Free package. Free software has the advantage problems can be fixed at the right place.

    @meadori
    Copy link
    Member

    meadori commented Feb 16, 2012

    When I try to build with the attached patch on the 2.7 branch many (if not all) of the modules fail to build:

    Failed to build these modules:
    _bisect _bsddb _codecs_cn
    _codecs_hk _codecs_iso2022 _codecs_jp
    _codecs_kr _codecs_tw _collections
    _csv _ctypes _ctypes_test
    _curses _curses_panel _elementtree
    _functools _hashlib _heapq
    _hotshot _io _json
    _locale _lsprof _multibytecodec
    _multiprocessing _random _socket
    _sqlite3 _ssl _struct
    _testcapi _tkinter array
    audioop binascii bz2
    cmath cPickle crypt
    cStringIO datetime dbm
    fcntl future_builtins gdbm
    grp itertools linuxaudiodev
    math mmap nis
    operator ossaudiodev parser
    pyexpat readline resource
    select spwd strop
    syslog termios time
    unicodedata zlib

    I see several warnings in the build output:

    *** WARNING: renaming "_codecs_iso2022" since importing it failed: build/lib.lin
    ux-x86_64-2.7/_codecs_iso2022.so: undefined symbol: PyExc_ValueError
    ..
    *** WARNING: renaming "linuxaudiodev" since importing it failed: build/lib.linux
    -x86_64-2.7/linuxaudiodev.so: undefined symbol: Py_Py3kWarningFlag
    ...
    *** WARNING: renaming "_tkinter" since importing it failed: build/lib.linux-x86_64-2.7/_tkinter.so: undefined symbol: _Py_ZeroStruct

    @meadori meadori added build The build process and cross-build and removed stdlib Python modules in the Lib dir labels Feb 16, 2012
    @jankratochvil
    Copy link
    Mannequin Author

    jankratochvil mannequin commented Feb 20, 2012

    There was a bug for non-shared build of Python, thanks.

    This patch is an improvement on its but it is not yet complete.
    One should also replace "-Xlinker -export-dynamic" by "-Wl,--dynamic-list" so that linking application with libpython is no longer intrusive.

    But I found out GDB does not need to be affected by this Bug. When I use -fvisibility=hidden for GDB .c files then "-Xlinker -export-dynamic" has no longer any negative effect (GDB was using only "-Wl,--dynamic-list" now which works without libpython but not with libpython). Therefore I no longer mind much about this Bug.

    Still at least:
    1771024 libpython2.7.so.1.0
    ->
    1757776 libpython2.7.so.1.0

    Even with clean tree the upstream build generates for me:
    Python build finished, but the necessary bits to build these modules were not found:
    _bsddb _ssl bsddb185
    dl imageop sunaudiodev

    (so sunaudiodev.c was changed without any testing of it)

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Feb 20, 2012

    The patch is fairly incomplete: it assumes that the compiler supports __attribute__((visibility())), but it really needs to check for that.

    Also, there is no chance that this can go into Python 2.7; retargetting for 3.3.

    @jankratochvil
    Copy link
    Mannequin Author

    jankratochvil mannequin commented Mar 3, 2015

    It even crashes applications due to pollution of dynamic symbols namespace by application symbols as seen in:
    https://bugzilla.redhat.com/show_bug.cgi?id=1198158

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @iritkatriel
    Copy link
    Member

    @jankratochvil Is this issue still relevant?

    @iritkatriel iritkatriel added the pending The issue will be closed if no feedback is provided label Aug 18, 2022
    @jankratochvil
    Copy link

    I am no longer involved with GDB. But I also think it has been fixed for python2->3:

    $ /usr/bin/python2-config --ldflags
    -lpython2.7 -ldl -lm -Xlinker -export-dynamic
    $ /usr/bin/python3-config --ldflags
     -L/usr/lib64  -lcrypt -ldl  -lm -lm 
    

    @iritkatriel
    Copy link
    Member

    Great, thank you.

    @iritkatriel iritkatriel closed this as not planned Won't fix, can't repro, duplicate, stale Aug 18, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    build The build process and cross-build pending The issue will be closed if no feedback is provided performance Performance or resource usage
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants