classification
Title: Use gnu_get_libc_version() in platform.libc_ver()?
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: jwilk, lemburg, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2018-12-03 16:02 by vstinner, last changed 2019-06-27 07:04 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 10891 merged vstinner, 2018-12-04 11:47
PR 10951 merged vstinner, 2018-12-05 21:03
PR 14418 merged vstinner, 2019-06-26 23:43
Messages (11)
msg330952 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-03 16:02
Currently, platform.libc_ver() opens Python binary file (ex: /usr/bin/python3) and looks for a string like "GLIBC-2.28".

Maybe gnu_get_libc_version() should be exposed in Python to get the version of the running glibc version? And use it if available, or fall back on parsing the binary file (as done currenetly) otherwise.

Example:

$ cat x.c 
#include <gnu/libc-version.h>
#include <stdlib.h>
#include <stdio.h>

int
main(int argc, char *argv[])
{
    printf("GNU libc version: %s\n", gnu_get_libc_version());
    printf("GNU libc release: %s\n", gnu_get_libc_release());
    exit(EXIT_SUCCESS);
}

$ ./x
GNU libc version: 2.28
GNU libc release: stable


I'm not sure if it's possible that Python is compiled with glibc but run with a different libc implementation?


--


Alternative: run a program to get the libc version which *might* be different than the libc version of Python if the libc is upgraded in the meanwhile (unlikely, but is technically possible on a server running for days):

$ ldd --version
ldd (GNU libc) 2.28
...

$ /lib64/libc.so.6 
GNU C Library (GNU libc) stable release version 2.28.
...

$ rpm -q glibc
glibc-2.28-17.fc29.x86_64
...

etc.


--


See also discussions on platform.libc_ver() performance:
https://github.com/python/cpython/pull/10868
msg330953 - (view) Author: Jakub Wilk (jwilk) Date: 2018-12-03 16:24
You can use confstr to get (running) glibc version:

>>> os.confstr('CS_GNU_LIBC_VERSION')
'glibc 2.28'
msg330956 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-03 16:38
> >>> os.confstr('CS_GNU_LIBC_VERSION')
> 'glibc 2.28'

That's cool because it doesn't require to write new C code ;-)
msg331036 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-12-04 12:03
Currently libc_ver() can be used for other executable. See issue26544 for discussion about libc_ver().
msg331041 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-04 13:12
> Currently libc_ver() can be used for other executable. See issue26544 for discussion about libc_ver().

Oh, my PR had a bug: it ignores executable. Fixed: it now only uses os.confstr() if the executable argument is not set.
msg331109 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-05 13:04
New changeset 476b113ed8531b9fbb0bd023a05eb3af21996600 by Victor Stinner in branch 'master':
bpo-35389: platform.libc_ver() uses os.confstr() (GH-10891)
https://github.com/python/cpython/commit/476b113ed8531b9fbb0bd023a05eb3af21996600
msg331117 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-05 14:26
> Quick benchmark on Fedora 29:
> python3 -m perf command ./python -S -c 'import platform; platform.libc_ver()'
> 94.9 ms +- 4.3 ms -> 33.2 ms +- 1.4 ms: 2.86x faster (-65%)

Oops, my benchmark in the commit message was wrong, it includes the startup time...

Correct benchmark says 44,538x faster, it's *WAY* better!

[regex] 56.1 ms +- 1.9 ms -> [confstr] 1.26 us +- 0.04 us: 44537.88x faster (-100%)
msg331124 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2018-12-05 14:52
Nice. I never liked the "parse the executable approach", but there wasn't anything better available at the time.
msg331126 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-05 14:59
> Nice. I never liked the "parse the executable approach", but there wasn't anything better available at the time.

Aha. Well, it's not perfect but it works and was fast enough (since libc_ver() is never used in performance critical code) :-)

I'm now curious and looked at the history of this feature.

"man confstr" says:

> _CS_GNU_LIBC_VERSION (GNU C library only; since glibc 2.3.2)

glibc 2.3.2 has been released in March 2003, so it's fine, we should get this constant in most "modern" Linux (using glibc) in 2018 :-)

man gnu_get_libc_version says:

> These functions first appeared in glibc in version 2.1.

glibc 2.1 has been released in Feb 1999. Using this function might provide even better compatibility but I'm not sure that it's worth it to use it. As I wrote, I prefer to not write a new function C, if os.confstr() can already be used in pure Python!

Sadly, these functions (confstr(_CS_GNU_LIBC_VERSION) / gnu_get_libc_version()) are specific to glibc. Sorry, I'm not interested to support other libc, I mostly care about Fedora, sorry :-)
msg331185 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-05 22:21
New changeset 848acf7249b5669d73d70a7cb6e5ab60689cf825 by Victor Stinner in branch 'master':
bpo-35389: test.pythoninfo logs platform.libc_ver (GH-10951)
https://github.com/python/cpython/commit/848acf7249b5669d73d70a7cb6e5ab60689cf825
msg346713 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-27 07:04
New changeset a719c8f4bd3cc5b1d98700c15c4a818f4d5617a4 by Victor Stinner in branch 'master':
bpo-35389: platform.platform() calls libc_ver() without executable (GH-14418)
https://github.com/python/cpython/commit/a719c8f4bd3cc5b1d98700c15c4a818f4d5617a4
History
Date User Action Args
2019-06-27 07:04:39vstinnersetmessages: + msg346713
2019-06-26 23:43:33vstinnersetpull_requests: + pull_request14231
2018-12-05 22:21:56vstinnersetmessages: + msg331185
2018-12-05 21:03:13vstinnersetpull_requests: + pull_request10201
2018-12-05 14:59:31vstinnersetmessages: + msg331126
2018-12-05 14:52:24lemburgsetnosy: + lemburg
messages: + msg331124
2018-12-05 14:26:48vstinnersetmessages: + msg331117
2018-12-05 13:53:18vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-12-05 13:04:57vstinnersetmessages: + msg331109
2018-12-04 13:12:14vstinnersetmessages: + msg331041
2018-12-04 12:03:18serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg331036
2018-12-04 11:47:51vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request10130
2018-12-03 16:38:33vstinnersetmessages: + msg330956
2018-12-03 16:24:04jwilksetnosy: + jwilk
messages: + msg330953
2018-12-03 16:02:13vstinnercreate