classification
Title: platform.libc_ver() returns incorrect version number
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Thomas.Waldmann, lemburg, njs, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-03-12 00:36 by Thomas.Waldmann, last changed 2018-06-13 17:51 by serhiy.storchaka.

Files
File name Uploaded Description Edit
results-20160709-224820.csv njs, 2016-07-10 06:00
Pull Requests
URL Status Linked Edit
PR 7684 open serhiy.storchaka, 2018-06-13 17:51
Messages (8)
msg261624 - (view) Author: Thomas Waldmann (Thomas.Waldmann) Date: 2016-03-12 00:36
platform.libc_ver() is trivially broken as it uses string comparison internally to determine the maximum libc version number (which is obviously broken as "2.9" > "2.10").
msg261648 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-03-12 13:34
True. At the time the code was written, this was not an issue :-)

Is the libc version information documented somewhere ? If so, we could probably add a better parser for it.
msg261649 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-03-12 13:35
Adding other Python versions as well, since this is a bug.
msg270073 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2016-07-10 06:00
We just ran into this in pip -- https://github.com/pypa/pip/pull/3836

I'd really recommend dropping the current "grovel through the binary doing a regex search" strategy -- it's incredibly error prone, and AFAICT doesn't really give any value. This code OTOH reliably lets you detect glibc and gives the exact version number with no fuss:
   https://github.com/pypa/pip/blob/master/pip/utils/glibc.py#L9

What about non-glibc systems? Unfortunately the current libc_ver() turns out not to work well for those either. Attached is a CSV file showing the return value of ~1.2 billion calls to platform.libc_ver() by the last 6 months of pip users. You can see that the current code basically never returns anything useful for non-glibc platforms. (The one exception is that it seems to be able to detect uclibc 0.9.32 and label it as "libc 0.9.32".)

Don't get me wrong: it'd be really really useful if there were some way to detect and distinguish between the common non-glibc libcs like musl/bionic/uclibc/..., but I'm not sure how to do that -- and unfortunately the current code definitely doesn't do the job :-(.
msg270080 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-07-10 10:37
At the time the code was written, libc and glibc were in wide spread use, so it's not surprising that it doesn't work well for other C libs.

Note that the routine returns the highest libc version number used and required by the executable (usually the Python interpreter). This does not necessarily correspond to the version installed on the system. The purpose of the function was to determine the minimum libc compatibility requirements of the executable.

The routine you quote uses ctypes and only works for glibc, so parsing needs to be kept around as fallback solution. It also returns the libc version that is currently used on the system; not necessarily the minimum version required, so semantics are different.
msg270089 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2016-07-10 15:14
> The purpose of the function was to determine the minimum libc compatibility requirements of the executable.

For what it's worth, I didn't know this, the pip authors obviously didn't know this, and after rereading the docs just now I still can't quite tell that this was intended. I'm not sure why one would want these semantics either, but at a minimum it would help to document the intended semantics much more clearly.

> parsing needs to be kept around as fallback solution

The point of the data I attached was that AFAICT there don't exist any currently-in-use, non-glibc systems where "parsing" returns a meaningful result.
msg319456 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-06-13 12:34
I need the glibc version for skipping a test if run with glibc containing a bug (see issue31630 and issue33630). platform.libc_ver() is not usable, it always returns '2.9' (the version that has bugs), while the actual version on my computer is '2.25' (the version that doesn't have these bugs).
msg319479 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-06-13 17:50
I agree that the strategy used for platform.libc_ver() is not perfect. But the implementation has bugs that make it useless. The following PR fixes two bugs in the implementation:

1) Version numbers compared as strings.
2) Versions that are located on the border of 16 KiB blocks were not recognized or were recognized incorrectly.
History
Date User Action Args
2018-06-13 17:51:27serhiy.storchakasetversions: + Python 3.7, Python 3.8, - Python 3.4, Python 3.5
2018-06-13 17:51:06serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request7297
2018-06-13 17:50:38serhiy.storchakasetmessages: + msg319479
2018-06-13 12:34:53serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg319456
2016-07-10 15:14:08njssetmessages: + msg270089
2016-07-10 10:37:00lemburgsetmessages: + msg270080
2016-07-10 06:00:48njssetfiles: + results-20160709-224820.csv
nosy: + njs
messages: + msg270073

2016-03-12 13:35:18lemburgsetmessages: + msg261649
versions: + Python 2.7, Python 3.4, Python 3.6
2016-03-12 13:34:39lemburgsetmessages: + msg261648
2016-03-12 00:39:49ned.deilysetnosy: + lemburg
2016-03-12 00:36:03Thomas.Waldmanncreate