classification
Title: In sysconfig, don't rely on sys.version format
Type: Stage: patch review
Components: Library (Lib) Versions:
process
Status: open Resolution:
Dependencies: 25985 Superseder:
Assigned To: Nosy List: benjamin.peterson, georg.brandl, lys.nikolaou, martin.panter, ned.deily, serhiy.storchaka, takluyver, vstinner
Priority: normal Keywords: patch

Created on 2015-08-22 17:52 by takluyver, last changed 2020-02-12 19:53 by vstinner.

Files
File name Uploaded Description Edit
sysconfig-version-fixme.patch takluyver, 2015-08-22 17:52 review
Pull Requests
URL Status Linked Edit
PR 10321 open lys.nikolaou, 2018-11-04 15:01
PR 18487 open damani, 2020-02-12 15:42
Messages (10)
msg248989 - (view) Author: Thomas Kluyver (takluyver) * Date: 2015-08-22 17:52
sysconfig currently calculates various formats of the Python version number by chopping up the sys.version string. This has a FIXME by it in the code, because the the format of sys.version is not guaranteed.

With this patch, the config variables 'py_version', 'py_version_short' and 'py_version_nodot' are instead generated from sys.version_info, which has a specified structure:
https://docs.python.org/3/library/sys.html#sys.version_info

One piece of information is lost by this change: after a pre-release, a + is added to the version string - e.g. '3.5.0b4+' means an unreleased version somewhere after 3.5.0b4. I can't find any structured representation of this information, so 'py_version' no longer contains it. I'm not sure whether it matters: I can't find anything using the 'py_version' config variable.
msg259911 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-02-09 02:49
Issue 25985 also touches the _PY_VERSION_SHORT parts of this.
msg260051 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-10 22:31
@Serhiy: Can you please take a look since this issue now depends on your issue #25985?
msg260060 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-02-10 23:30
I just added the dependency to reflect that the patch here will need updating.

The main concern here is Thomas’s question: does it matter that py_version loses prerelease info?
msg260073 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-02-11 04:42
My understanding is that the "+" is added to the PY_VERSION in Include/patchlevel.h by the release management process after any tagged release, whether pre-release or final.  So '+" is supposed to be set whenever CPython is built from anything other than an official tagged revision.  AFAICT, "py_version" and friends were added to sysconfig when sysconfig was initially moved out of distutils (in fa69e891edf4) to become its own standalone module as part of the last big set of distutils enhancements which were later largely reverted, ending up with two versions of sysconfig: the newer standalone sysconfig.py and with the older distutils/sysconfig.py.  It appears "py_version" and friends have never been implemented in the distutils/sysconfig.py so it's likely that they aren't used much in the wild but it would be nice to not break the current compatibility.  Perhaps expanding sys.version_info to contain the "modified" (?) ("+") field would be a good idea or possibly adding a new value to "releaselevel".  As it currently stands:

$ /usr/local/bin/python3.5
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.version
'3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44) \n[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]'
>>> sys.version_info
sys.version_info(major=3, minor=5, micro=1, releaselevel='final', serial=0)

$ ./python
Python 3.5.1+ (default, Feb 11 2016, 14:00:02)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=5, micro=1, releaselevel='final', serial=0)
>>> sys.version
'3.5.1+ (default, Feb 11 2016, 14:00:02) \n[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]'

It seems wrong that one cannot use sys.version_info to distinguish between a release build and a post-release development build.  

I'm nosying Georg and Benjamin for institutional memory picking.
msg260092 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-02-11 11:37
Most changes were committed in issue25985. Only _PY_VERSION is left. I have no strong opinion about this, but +0 for keeping "+".
msg328779 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-29 00:47
Noted in passing: while Lib/distutils/sysconfig.py has not implemented "py_version", I see now that Lib/distutils/command/install.py does have something very similar to Lib/sysconfig.py so whatever (if anything) is changed in one should also be changed in the other.
msg329011 - (view) Author: Lysandros Nikolaou (lys.nikolaou) * Date: 2018-10-31 20:09
I'm working on changing _PY_VERSION to get its value from sys.version_info instead of sys.version, but I am not sure what the best way is to map between a release stage to the corresponding letter(release->rc). Could someone help me a tiny bit with this one?
msg329265 - (view) Author: Lysandros Nikolaou (lys.nikolaou) * Date: 2018-11-04 21:48
Following up on https://github.com/python/cpython/pull/10321#discussion_r230604393 I would like to summarise here what's been going on, in order to move the discussion here forward. I've tried to make a PR for this issue, in which _PY_VERSION in Lib/sysconfig.py and py_version in Lib/distutils/command/install.py are updated to get their value from sys.version_info instead of sys.version. This PR removes the '+' from both so the issue remains, if we want to keep the '+' info or not.
msg361910 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-02-12 19:53
The "py_version" variable of sysconfig is discussed since 2015 but... it's unused. It is supposed to build paths. Does it really make sense to build a path which contains "+"? It can confuse some application which may associate a special meaning to "+".

What about simply removing this variable? Does anyone know if it's used in the wild?

To me it sounds surprising to have a different path for alpha, beta and release candidate releases. Usually, Python paths only contain MAJOR.MINOR versions, no alpha, beta or rc marker. Nor "+" marker.
History
Date User Action Args
2020-02-12 19:53:01vstinnersetmessages: + msg361910
2020-02-12 15:42:42damanisetpull_requests: + pull_request17860
2018-11-04 21:48:42lys.nikolaousetmessages: + msg329265
2018-11-04 15:01:25lys.nikolaousetpull_requests: + pull_request9623
2018-10-31 20:09:39lys.nikolaousetnosy: + lys.nikolaou
messages: + msg329011
2018-10-29 00:47:17ned.deilysetmessages: + msg328779
2018-10-29 00:42:50ned.deilylinkissue35096 superseder
2017-04-18 07:22:14serhiy.storchakalinkissue30092 superseder
2016-02-11 11:37:19serhiy.storchakasetmessages: + msg260092
2016-02-11 04:42:57ned.deilysetnosy: + ned.deily, benjamin.peterson, georg.brandl
messages: + msg260073
2016-02-10 23:30:55martin.pantersetmessages: + msg260060
2016-02-10 22:31:55vstinnersetnosy: + vstinner, serhiy.storchaka
messages: + msg260051
2016-02-09 02:49:40martin.pantersetnosy: + martin.panter
messages: + msg259911

dependencies: + Use sys.version_info instead of sys.version
stage: patch review
2015-08-22 17:52:34takluyvercreate