classification
Title: Deprecate slicing and ordering operations on sys.version
Type: enhancement Stage: needs patch
Components: Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Anthony Sottile, Philip Dye, ezio.melotti, josh.r, ncoghlan, p-ganssle, rhettinger, rkm, scoder, serhiy.storchaka
Priority: normal Keywords:

Created on 2019-07-20 12:45 by ncoghlan, last changed 2020-03-02 15:14 by rkm.

Messages (11)
msg348213 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2019-07-20 12:45
At the core dev sprints in 2017 (IIRC), Ezio and I were talking about some of the potential issues with 3.10, and one of them was folks attempting to manipulate sys.version with string operations rather than using the already decomposed sys.version info.

Anthony Sottile has recently been experimenting with a Python build patched to report itself as 3.10 to see what breaks, and indicated that many of the failures were due to that kind of problem (in particular: writing "sys.version[:3]" instead of "'{}.{}'.format(sys.version_info[:2])")

One possible remedy that Ezio and I discussed was the idea of an "OpaqueString" type that presented as a string, but raised an error if you tried any operations that assumed it could be meaningfully decomposed character-by-character.

This would probably need to be a string subclass for compatibility, and would need a transitional variant that only emitted deprecation warnings rather than failing outright.
msg348217 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-07-20 17:09
Have we considered just rolling our next version to 4.0 instead of 3.10?
msg348219 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2019-07-20 19:16
Rolling to 4.0 implies backward incompatible changes; doing just to avoid 3.10 is breaking semantic versioning rules.
msg348220 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-07-20 20:26
> Rolling to 4.0 implies backward incompatible changes

It might or it might not depending on how we communicate it.  See Python 2.0 for a precedent.  That was not a backward incompatible release.
msg348221 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-07-20 21:08
Changing the major version number itself is a breaking change. For example, there is a code that checks sys.version_info[0] == 3 or even sys.version[0] == "3".
msg348236 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2019-07-21 11:11
> Changing the major version number itself is a breaking change

So is the proposed change, in a way. At some point, there will be a 4.0 release, which may or may not break the code in question. So, testing for "version_info[0] == 3" is already wrong, because that code almost certainly wants to know if it runs in Py2 and not if it runs in Py3. It's better fixed soon, before we get too close to Py4, and I think a deprecation notice could be helpful for that. Preferably for as many of the non-future-proof spellings as possible.
msg348240 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-07-21 12:19
I think it is better to left this on third-party linters.

* It is easy to catch patterns like `sys.version_info[0] == 3` or `sys.version[:3]` in the linter than in the Python compiler or interpreter. Linters are designed for this.

* Not always slicing sys.version is an error. For example, `sys.version[:sys.version.index(' ')]` is a valid code. Linters have options which allow to ignore some kind of warnings or warnings at the particular line.

* Users that ignore linters usually do not check the version. They wrote their scripts for the default system Python and update them when they become broken after system upgrade.
msg348241 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2019-07-21 12:31
> So is the proposed change, in a way. At some point, there will be a 4.0 release, which may or may not break the code in question.

I don't think it's a foregone conclusion that there will be a 4.0 release. It could be that we decide that a 4.0 release would be reserved for a very specific kind of major compatibility breakage and that we were never going to do that kind of breakage again.

In any case, I think one part of what Serhiy was trying to say is that we can't just avoid this problem by releasing 4.0 instead of 3.10, because switching to 4.0 brings its own (not inconsiderable) host of problems. Presumably the symlink would be `python4` instead of `python3`, so a bunch of shebangs, documentation and guides would need to be updated.

Basically, unless something changes, a bunch of stuff is going to break in the release after 3.9 no matter what we do. It's probably a good idea to try to mitigate the problems of *both* versioning approaches, to give us maximum freedom in our choice of versioning scheme int he future.
msg349296 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2019-08-09 17:00
Regarding the "Would releasing 4.0 instead avoid problems?" Anthony actually did that experiment, too: it broke third party projects even more impressively than the 3.10 build did.

I think Serhiy's point is fairly compelling though - check whether or not pylint checks for this, and then close this issue with either a pointer to their documentation, or else a pointer to a feature request on their issue tracker.
msg349332 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-10 05:50
> Regarding the "Would releasing 4.0 instead avoid problems?"
> Anthony actually did that experiment, too: it broke third
> party projects even more impressively than the 3.10 build did.

That's good to know.  Please thank Anthony for doing his homework.
msg350413 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2019-08-25 00:14
I threw together a flake8 plugin which checks for these usage patterns:

https://github.com/asottile/flake8-2020


| Code   | Description                                     |
|--------|-------------------------------------------------|
| YTT101 | `sys.version[:...]` referenced (python3.10)     |
| YTT102 | `sys.version[2]` referenced (python3.10)        |
| YTT201 | `sys.version_info[0] == 3` referenced (python4) |
| YTT202 | `six.PY3` referenced (python4)                  |
| YTT301 | `sys.version[0]` referenced (python10)          |
History
Date User Action Args
2020-03-02 15:14:59rkmsetnosy: + rkm
2019-08-25 00:14:20Anthony Sottilesetmessages: + msg350413
2019-08-10 05:50:27rhettingersetmessages: + msg349332
2019-08-09 17:00:31ncoghlansetmessages: + msg349296
2019-08-08 21:33:12Philip Dyesetnosy: + Philip Dye
2019-07-21 12:31:07p-gansslesetnosy: + p-ganssle
messages: + msg348241
2019-07-21 12:19:29serhiy.storchakasetmessages: + msg348240
2019-07-21 11:11:51scodersetnosy: + scoder
messages: + msg348236
2019-07-20 21:08:26serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg348221
2019-07-20 20:26:45rhettingersetmessages: + msg348220
2019-07-20 19:16:36josh.rsetnosy: + josh.r
messages: + msg348219
2019-07-20 17:09:33rhettingersetnosy: + rhettinger
messages: + msg348217
2019-07-20 14:06:26Anthony Sottilesetnosy: + Anthony Sottile
2019-07-20 12:45:55ncoghlancreate