classification
Title: Update old Python 2 code in Docs/tools/extensions/patchlevel.py
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Jim Fasarakis-Hilliard, db3l, docs@python, martin.panter, ned.deily, python-dev
Priority: normal Keywords: patch

Created on 2017-01-23 09:41 by Jim Fasarakis-Hilliard, last changed 2017-03-31 16:36 by dstufft. This issue is now closed.

Files
File name Uploaded Description Edit
patchlevel.patch Jim Fasarakis-Hilliard, 2017-01-23 09:41 review
patchlevel_print.patch Jim Fasarakis-Hilliard, 2017-01-24 11:59 review
patchlevel_with.patch Jim Fasarakis-Hilliard, 2017-01-24 12:00 review
Pull Requests
URL Status Linked Edit
PR 552 closed dstufft, 2017-03-31 16:36
Messages (14)
msg286065 - (view) Author: Jim Fasarakis-Hilliard (Jim Fasarakis-Hilliard) * Date: 2017-01-23 09:41
There's a hidden Python 2 print call in the script that only gets reached after you move the Doc/ folder outside the main CPython directory and run `make html`. 

Additionally, an obsolete way of assuring a file gets closed is used (changed to use `with` statement).

P.S: Tagged this with the Documentation component, maybe build is more applicable but I wasn't sure.
msg286136 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-01-24 05:02
Is it okay to only fix this in 3.5+? 3.4 only gets security fixes now.

Either way, the “with” statement changes is not a bug fix and should only go into 3.7.
msg286178 - (view) Author: Jim Fasarakis-Hilliard (Jim Fasarakis-Hilliard) * Date: 2017-01-24 11:59
I'm breaking these to separate files to make it easier to apply. I also noticed that other files in `Doc/tools/extensions/` use old constructs so I'm not sure about the *with*.

I'm guessing that either it should be changed in other files too or, since it's working fine, leave it as is.
msg286438 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-01-29 10:34
New changeset ecdc864884a9 by Martin Panter in branch '3.5':
Issue #29349: Fix Python 2 syntax in documentation build code
https://hg.python.org/cpython/rev/ecdc864884a9

New changeset cdcb33f37bf3 by Martin Panter in branch '3.6':
Issues #29349: Merge Py 2 fix 3.5
https://hg.python.org/cpython/rev/cdcb33f37bf3

New changeset db8b917bbfdd by Martin Panter in branch 'default':
Issues #29349: Merge Py 2 fix 3.6
https://hg.python.org/cpython/rev/db8b917bbfdd

New changeset ca7d2af9920e by Martin Panter in branch 'default':
Issues #29349: Add NEWS for 3.7; use “with” statement
https://hg.python.org/cpython/rev/ca7d2af9920e
msg286440 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-01-29 10:53
I think the general rule is to clean up code if you are doing something else in nearby code, but don’t go out of your way with unnecessary cleanups to arbitrary code. Otherwise it adds too much noise to the repository history, review process, risks adding bugs, etc, for little gain.

Anyway, thanks for the patch!
msg286445 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-01-29 17:11
These changes have broken some buildbots, for example, the OS X dmg buildbots which are still using Python 2 for sphinx builds of the docs.  They should be version agnostic as much as possible.

http://buildbot.python.org/all/builders/bolen-dmg-3.x/builds/59
msg286450 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-01-29 21:33
Thanks Ned. Do you know what version of Python Sphinx uses (which runs patchlevel.py)?

According to Issue 28039, David set up Python 2.7 so that “make touch” would work. But the log also uses a python2.5 command, and apparently Python 2.3 also installed on that buildbot.

If we can rely on 2.6+, we just need to add “from __future__ import print_function”. Otherwise, we may need something like

if sys.stderr is not None:
    sys.stderr.write(...)

and also do something about that “with” statement.
msg286451 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-01-29 21:39
Dunno for sure, perhaps David can answer that.  But the Sphinx docs imply that Sphinx 1.4.6 requires at least 2.6 and I don't test the installer build with anything less than 2.7.
msg286452 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-01-29 21:40
According to <https://bugs.python.org/issue17861#msg217417>, Ned says 2.6+ is already needed to build the Python 3.5 documentation, so maybe Sphinx uses that.
msg286454 - (view) Author: David Bolen (db3l) Date: 2017-01-29 22:28
2.7.12 (/usr/local/bin) is used for the build slave and main external commands (such as hg and sphinx) and is in the buildslave path, but 2.5.1 is still the default in /usr/bin so can get used for processes with a restricted environment.  Tiger's original 2.3 is present but unused.

The DMG build-installer script itself runs under 2.5.1, because it specifies "python2.5" in its command to the slave.  I believe that was originally added once the system 2.3 couldn't be used.

During the DMG construction process, a particular step might, I suppose use 2.5.1 or 2.7.12 depending on whether it's internal to the build script, something using the same executable as the script, a subprocess with limited path, or an independent utility (like sphinx) referencing 2.7.

I believe Ned's comment in issue 17861 about 2.6+ is a reference to the sphinx step, which is covered by the fact that sphinx is using the newer 2.7.12.

What appears to break in this specific case is some of the preparation - specifically the Makefile referencing patchlevel is running "python".  That should work (2.7.12) if it inherits the overall build slave environment (which has /usr/local/bin first) but I think the install script filters that, so it's probably getting the /usr/bin/python 2.5.1 version.

I've just changed the default /usr/bin/python to be 2.7 so external processes run by build-installer should get 2.7.  I don't think that will break anything else currently done on the machine, but I'll deal with any fallout if needed.  I'm rerunning build 59 now.

As in issue 28039 there's no immediate need to change the installer script itself, but since it's clearly the last hold out, I'll find some time to test that I've got any pre-requisites for it set up at which point the build master could be changed to remove the python2.5 reference.
msg286455 - (view) Author: David Bolen (db3l) Date: 2017-01-29 22:38
Whoops, I just realized that the patch still needs adjusting to be 2.x compatible, so obviously the extra build still won't work.

But at this point it should be safe to assume 2.6+ such as for the rest of the sphinx processing.
msg286458 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-01-29 23:48
New changeset 9880928ec962 by Martin Panter in branch '3.5':
Issue #29349: Use __future__ print_function; Sphinx may use Python 2.6+
https://hg.python.org/cpython/rev/9880928ec962

New changeset 1708afd284ff by Martin Panter in branch '3.6':
Issues #29349: Merge Py 2.6+ compatibility from 3.5
https://hg.python.org/cpython/rev/1708afd284ff

New changeset e50058ecd808 by Martin Panter in branch 'default':
Issues #29349: Merge Py 2.6+ compatibility from 3.6
https://hg.python.org/cpython/rev/e50058ecd808
msg286461 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-01-29 23:57
I pushed the simpler 2.6-compatible option. Keeping this open to check the buildbot is happy overnight.
msg286496 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-01-30 18:08
The fix looks good and the dmg buildbots are happy again.  Thanks, Martin and David.
History
Date User Action Args
2017-03-31 16:36:27dstufftsetpull_requests: + pull_request1007
2017-01-30 18:08:11ned.deilysetstatus: open -> closed
resolution: fixed
messages: + msg286496
2017-01-29 23:57:29martin.pantersetmessages: + msg286461
stage: needs patch -> resolved
2017-01-29 23:48:17python-devsetmessages: + msg286458
2017-01-29 22:38:09db3lsetmessages: + msg286455
2017-01-29 22:28:46db3lsetmessages: + msg286454
2017-01-29 21:40:29martin.pantersetmessages: + msg286452
2017-01-29 21:39:50ned.deilysetmessages: + msg286451
2017-01-29 21:33:12martin.pantersetnosy: + db3l
messages: + msg286450
2017-01-29 17:11:40ned.deilysetstatus: closed -> open

nosy: + ned.deily
messages: + msg286445

resolution: fixed -> (no value)
stage: resolved -> needs patch
2017-01-29 10:53:31martin.pantersetstatus: open -> closed
versions: - Python 3.4
messages: + msg286440

resolution: fixed
stage: commit review -> resolved
2017-01-29 10:34:26python-devsetnosy: + python-dev
messages: + msg286438
2017-01-24 12:00:03Jim Fasarakis-Hilliardsetfiles: + patchlevel_with.patch
2017-01-24 11:59:55Jim Fasarakis-Hilliardsetfiles: + patchlevel_print.patch

messages: + msg286178
2017-01-24 05:02:41martin.pantersetnosy: + martin.panter

messages: + msg286136
stage: commit review
2017-01-23 09:41:21Jim Fasarakis-Hilliardcreate