Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use gnu_get_libc_version() in platform.libc_ver()? #79570

Closed
vstinner opened this issue Dec 3, 2018 · 11 comments
Closed

Use gnu_get_libc_version() in platform.libc_ver()? #79570

vstinner opened this issue Dec 3, 2018 · 11 comments
Labels
3.8 only security fixes stdlib Python modules in the Lib dir

Comments

@vstinner
Copy link
Member

vstinner commented Dec 3, 2018

BPO 35389
Nosy @malemburg, @vstinner, @jwilk, @serhiy-storchaka
PRs
  • bpo-35389: platform.libc_ver() uses os.confstr() #10891
  • bpo-35389: test.pythoninfo logs platform.libc_ver #10951
  • bpo-35389: platform.platform() calls libc_ver() without executable #14418
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2018-12-05.13:53:18.228>
    created_at = <Date 2018-12-03.16:02:13.435>
    labels = ['3.8', 'library']
    title = 'Use gnu_get_libc_version() in platform.libc_ver()?'
    updated_at = <Date 2019-06-27.07:04:39.140>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2019-06-27.07:04:39.140>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2018-12-05.13:53:18.228>
    closer = 'vstinner'
    components = ['Library (Lib)']
    creation = <Date 2018-12-03.16:02:13.435>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 35389
    keywords = ['patch']
    message_count = 11.0
    messages = ['330952', '330953', '330956', '331036', '331041', '331109', '331117', '331124', '331126', '331185', '346713']
    nosy_count = 4.0
    nosy_names = ['lemburg', 'vstinner', 'jwilk', 'serhiy.storchaka']
    pr_nums = ['10891', '10951', '14418']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue35389'
    versions = ['Python 3.8']

    @vstinner
    Copy link
    Member Author

    vstinner commented Dec 3, 2018

    Currently, platform.libc_ver() opens Python binary file (ex: /usr/bin/python3) and looks for a string like "GLIBC-2.28".

    Maybe gnu_get_libc_version() should be exposed in Python to get the version of the running glibc version? And use it if available, or fall back on parsing the binary file (as done currenetly) otherwise.

    Example:

    $ cat x.c 
    #include <gnu/libc-version.h>
    #include <stdlib.h>
    #include <stdio.h>
    
    int
    main(int argc, char *argv[])
    {
        printf("GNU libc version: %s\n", gnu_get_libc_version());
        printf("GNU libc release: %s\n", gnu_get_libc_release());
        exit(EXIT_SUCCESS);
    }
    $ ./x
    GNU libc version: 2.28
    GNU libc release: stable

    I'm not sure if it's possible that Python is compiled with glibc but run with a different libc implementation?

    --

    Alternative: run a program to get the libc version which *might* be different than the libc version of Python if the libc is upgraded in the meanwhile (unlikely, but is technically possible on a server running for days):

    $ ldd --version
    ldd (GNU libc) 2.28
    ...
    
    $ /lib64/libc.so.6 
    GNU C Library (GNU libc) stable release version 2.28.
    ...
    
    $ rpm -q glibc
    glibc-2.28-17.fc29.x86_64
    ...

    etc.

    --

    See also discussions on platform.libc_ver() performance:
    #10868

    @vstinner vstinner added 3.8 only security fixes stdlib Python modules in the Lib dir labels Dec 3, 2018
    @jwilk
    Copy link
    Mannequin

    jwilk mannequin commented Dec 3, 2018

    You can use confstr to get (running) glibc version:

    >>> os.confstr('CS_GNU_LIBC_VERSION')
    'glibc 2.28'

    @vstinner
    Copy link
    Member Author

    vstinner commented Dec 3, 2018

    >>> os.confstr('CS_GNU_LIBC_VERSION')
    'glibc 2.28'

    That's cool because it doesn't require to write new C code ;-)

    @serhiy-storchaka
    Copy link
    Member

    Currently libc_ver() can be used for other executable. See bpo-26544 for discussion about libc_ver().

    @vstinner
    Copy link
    Member Author

    vstinner commented Dec 4, 2018

    Currently libc_ver() can be used for other executable. See bpo-26544 for discussion about libc_ver().

    Oh, my PR had a bug: it ignores executable. Fixed: it now only uses os.confstr() if the executable argument is not set.

    @vstinner
    Copy link
    Member Author

    vstinner commented Dec 5, 2018

    New changeset 476b113 by Victor Stinner in branch 'master':
    bpo-35389: platform.libc_ver() uses os.confstr() (GH-10891)
    476b113

    @vstinner vstinner closed this as completed Dec 5, 2018
    @vstinner
    Copy link
    Member Author

    vstinner commented Dec 5, 2018

    Quick benchmark on Fedora 29:
    python3 -m perf command ./python -S -c 'import platform; platform.libc_ver()'
    94.9 ms +- 4.3 ms -> 33.2 ms +- 1.4 ms: 2.86x faster (-65%)

    Oops, my benchmark in the commit message was wrong, it includes the startup time...

    Correct benchmark says 44,538x faster, it's *WAY* better!

    [regex] 56.1 ms +- 1.9 ms -> [confstr] 1.26 us +- 0.04 us: 44537.88x faster (-100%)

    @malemburg
    Copy link
    Member

    Nice. I never liked the "parse the executable approach", but there wasn't anything better available at the time.

    @vstinner
    Copy link
    Member Author

    vstinner commented Dec 5, 2018

    Nice. I never liked the "parse the executable approach", but there wasn't anything better available at the time.

    Aha. Well, it's not perfect but it works and was fast enough (since libc_ver() is never used in performance critical code) :-)

    I'm now curious and looked at the history of this feature.

    "man confstr" says:

    _CS_GNU_LIBC_VERSION (GNU C library only; since glibc 2.3.2)

    glibc 2.3.2 has been released in March 2003, so it's fine, we should get this constant in most "modern" Linux (using glibc) in 2018 :-)

    man gnu_get_libc_version says:

    These functions first appeared in glibc in version 2.1.

    glibc 2.1 has been released in Feb 1999. Using this function might provide even better compatibility but I'm not sure that it's worth it to use it. As I wrote, I prefer to not write a new function C, if os.confstr() can already be used in pure Python!

    Sadly, these functions (confstr(_CS_GNU_LIBC_VERSION) / gnu_get_libc_version()) are specific to glibc. Sorry, I'm not interested to support other libc, I mostly care about Fedora, sorry :-)

    @vstinner
    Copy link
    Member Author

    vstinner commented Dec 5, 2018

    New changeset 848acf7 by Victor Stinner in branch 'master':
    bpo-35389: test.pythoninfo logs platform.libc_ver (GH-10951)
    848acf7

    @vstinner
    Copy link
    Member Author

    New changeset a719c8f by Victor Stinner in branch 'master':
    bpo-35389: platform.platform() calls libc_ver() without executable (GH-14418)
    a719c8f

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants