classification
Title: In sysconfig, don't rely on sys.version format
Type: Stage: resolved
Components: Library (Lib) Versions:
process
Status: closed Resolution: out of date
Dependencies: 25985 Superseder:
Assigned To: Nosy List: benjamin.peterson, georg.brandl, lys.nikolaou, martin.panter, ned.deily, serhiy.storchaka, takluyver, vstinner
Priority: normal Keywords: patch

Created on 2015-08-22 17:52 by takluyver, last changed 2020-03-22 00:23 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
sysconfig-version-fixme.patch takluyver, 2015-08-22 17:52 review
Pull Requests
URL Status Linked Edit
PR 10321 closed lys.nikolaou, 2018-11-04 15:01
PR 18487 closed damani, 2020-02-12 15:42
PR 19101 merged serhiy.storchaka, 2020-03-21 11:52
Messages (14)
msg248989 - (view) Author: Thomas Kluyver (takluyver) * Date: 2015-08-22 17:52
sysconfig currently calculates various formats of the Python version number by chopping up the sys.version string. This has a FIXME by it in the code, because the the format of sys.version is not guaranteed.

With this patch, the config variables 'py_version', 'py_version_short' and 'py_version_nodot' are instead generated from sys.version_info, which has a specified structure:
https://docs.python.org/3/library/sys.html#sys.version_info

One piece of information is lost by this change: after a pre-release, a + is added to the version string - e.g. '3.5.0b4+' means an unreleased version somewhere after 3.5.0b4. I can't find any structured representation of this information, so 'py_version' no longer contains it. I'm not sure whether it matters: I can't find anything using the 'py_version' config variable.
msg259911 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-02-09 02:49
Issue 25985 also touches the _PY_VERSION_SHORT parts of this.
msg260051 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-10 22:31
@Serhiy: Can you please take a look since this issue now depends on your issue #25985?
msg260060 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-02-10 23:30
I just added the dependency to reflect that the patch here will need updating.

The main concern here is Thomas’s question: does it matter that py_version loses prerelease info?
msg260073 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-02-11 04:42
My understanding is that the "+" is added to the PY_VERSION in Include/patchlevel.h by the release management process after any tagged release, whether pre-release or final.  So '+" is supposed to be set whenever CPython is built from anything other than an official tagged revision.  AFAICT, "py_version" and friends were added to sysconfig when sysconfig was initially moved out of distutils (in fa69e891edf4) to become its own standalone module as part of the last big set of distutils enhancements which were later largely reverted, ending up with two versions of sysconfig: the newer standalone sysconfig.py and with the older distutils/sysconfig.py.  It appears "py_version" and friends have never been implemented in the distutils/sysconfig.py so it's likely that they aren't used much in the wild but it would be nice to not break the current compatibility.  Perhaps expanding sys.version_info to contain the "modified" (?) ("+") field would be a good idea or possibly adding a new value to "releaselevel".  As it currently stands:

$ /usr/local/bin/python3.5
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.version
'3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44) \n[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]'
>>> sys.version_info
sys.version_info(major=3, minor=5, micro=1, releaselevel='final', serial=0)

$ ./python
Python 3.5.1+ (default, Feb 11 2016, 14:00:02)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=5, micro=1, releaselevel='final', serial=0)
>>> sys.version
'3.5.1+ (default, Feb 11 2016, 14:00:02) \n[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]'

It seems wrong that one cannot use sys.version_info to distinguish between a release build and a post-release development build.  

I'm nosying Georg and Benjamin for institutional memory picking.
msg260092 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-02-11 11:37
Most changes were committed in issue25985. Only _PY_VERSION is left. I have no strong opinion about this, but +0 for keeping "+".
msg328779 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-29 00:47
Noted in passing: while Lib/distutils/sysconfig.py has not implemented "py_version", I see now that Lib/distutils/command/install.py does have something very similar to Lib/sysconfig.py so whatever (if anything) is changed in one should also be changed in the other.
msg329011 - (view) Author: Lysandros Nikolaou (lys.nikolaou) * (Python committer) Date: 2018-10-31 20:09
I'm working on changing _PY_VERSION to get its value from sys.version_info instead of sys.version, but I am not sure what the best way is to map between a release stage to the corresponding letter(release->rc). Could someone help me a tiny bit with this one?
msg329265 - (view) Author: Lysandros Nikolaou (lys.nikolaou) * (Python committer) Date: 2018-11-04 21:48
Following up on https://github.com/python/cpython/pull/10321#discussion_r230604393 I would like to summarise here what's been going on, in order to move the discussion here forward. I've tried to make a PR for this issue, in which _PY_VERSION in Lib/sysconfig.py and py_version in Lib/distutils/command/install.py are updated to get their value from sys.version_info instead of sys.version. This PR removes the '+' from both so the issue remains, if we want to keep the '+' info or not.
msg361910 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-02-12 19:53
The "py_version" variable of sysconfig is discussed since 2015 but... it's unused. It is supposed to build paths. Does it really make sense to build a path which contains "+"? It can confuse some application which may associate a special meaning to "+".

What about simply removing this variable? Does anyone know if it's used in the wild?

To me it sounds surprising to have a different path for alpha, beta and release candidate releases. Usually, Python paths only contain MAJOR.MINOR versions, no alpha, beta or rc marker. Nor "+" marker.
msg364156 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-03-14 12:03
What is the problem with the current code?
msg364413 - (view) Author: Thomas Kluyver (takluyver) * Date: 2020-03-17 11:33
Serhiy, I think you fixed the part that was actually likely to cause problems a few years ago in issue #25985 & commit 885bdc4946890f4bb80557fab80c3874b2cc4d39 . Using sys.version[:3] to get a short version like 3.8 was what I wanted to fix.

People are presumably still trying to change the other bits because there's still a FIXME comment. If we're happy with the current code, I think we can remove that comment and close the issue.
msg364746 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-03-21 13:45
New changeset 684d2b9a071fa8e54749e0eec3c16aafcd642ed4 by Serhiy Storchaka in branch 'master':
bpo-24916: Remove an outdated comment. (GH-19101)
https://github.com/python/cpython/commit/684d2b9a071fa8e54749e0eec3c16aafcd642ed4
msg364779 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-22 00:23
Since this issue has been closed, I closed PR 10321 and PR 18487.
History
Date User Action Args
2020-03-22 00:23:20vstinnersetnosy: + vstinner
messages: + msg364779
2020-03-21 13:46:40serhiy.storchakasetstatus: open -> closed
resolution: out of date
stage: patch review -> resolved
2020-03-21 13:45:44serhiy.storchakasetmessages: + msg364746
2020-03-21 11:52:16serhiy.storchakasetpull_requests: + pull_request18461
2020-03-17 16:55:57vstinnersetnosy: - vstinner
2020-03-17 11:33:57takluyversetmessages: + msg364413
2020-03-14 12:03:16serhiy.storchakasetmessages: + msg364156
2020-02-12 19:53:01vstinnersetmessages: + msg361910
2020-02-12 15:42:42damanisetpull_requests: + pull_request17860
2018-11-04 21:48:42lys.nikolaousetmessages: + msg329265
2018-11-04 15:01:25lys.nikolaousetpull_requests: + pull_request9623
2018-10-31 20:09:39lys.nikolaousetnosy: + lys.nikolaou
messages: + msg329011
2018-10-29 00:47:17ned.deilysetmessages: + msg328779
2018-10-29 00:42:50ned.deilylinkissue35096 superseder
2017-04-18 07:22:14serhiy.storchakalinkissue30092 superseder
2016-02-11 11:37:19serhiy.storchakasetmessages: + msg260092
2016-02-11 04:42:57ned.deilysetnosy: + ned.deily, benjamin.peterson, georg.brandl
messages: + msg260073
2016-02-10 23:30:55martin.pantersetmessages: + msg260060
2016-02-10 22:31:55vstinnersetnosy: + vstinner, serhiy.storchaka
messages: + msg260051
2016-02-09 02:49:40martin.pantersetnosy: + martin.panter
messages: + msg259911

dependencies: + Use sys.version_info instead of sys.version
stage: patch review
2015-08-22 17:52:34takluyvercreate