Title: Bring Doc/make.bat as close to Doc/Makefile as possible
Type: enhancement Stage: resolved
Components: Build, Documentation, Windows Versions: Python 3.5, Python 3.4
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zach.ware Nosy List: brian.curtin, christian.heimes, docs@python, ezio.melotti, python-dev, terry.reedy, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2013-03-08 22:33 by zach.ware, last changed 2014-04-29 14:47 by python-dev. This issue is now closed.

File name Uploaded Description Edit
win_doc_make.diff zach.ware, 2013-03-08 22:33 Doc/make.bat overhaul review
win_doc_make_minipatch.diff zach.ware, 2013-03-09 20:36 Fix to %PYTHON% assignment review
win_doc_make.v2.diff zach.ware, 2013-03-15 16:58 Version 2 review
win_doc_make.v3.diff zach.ware, 2013-04-12 20:27 Version 3 review
win_doc_make.v4.diff zach.ware, 2013-04-15 21:25 Version 4 review
win_doc_make.v5.diff zach.ware, 2013-04-23 19:07 Version 5 review
issue17386.diff zach.ware, 2014-04-23 16:32 New approach review
issue17386.v2.diff zach.ware, 2014-04-28 21:05 review
Messages (16)
msg183768 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-03-08 22:33
I have found Doc/make.bat to be very useful, but I have hit a few of its limitations now and then.  Thus, I have made extensive modifications to bring it much closer to the way Doc/Makefile works.  Among the things changed in the attached patch:

- Use pushd/popd to make sure everything is done in the Doc directory.

- Allow variables to be set on the command line, e.g.:
       make html "PYTHON2=python"

- Set PYTHON2 and PYTHON3 variables separately; the Sphinx version and associated tools we're currently using want Python 2, which defaults to "py -2".  Other tools, like Tools/scripts/ (the make serve target), expect Python 3; setting the default PYTHON3 searches for python(_d).exe in PCbuild(\amd64), and falls back to "py -3" if not found.

- Set the HTML Help Workshop default path to %ProgramFiles(x86)%\..., if that variable is defined. (If I understand correctly, that variable is only defined on x64 installations, and in cases where it is not defined on x64, %ProgramFiles% silently copies %ProgramFiles(x86)%.)

- Convert target choice to a single for loop, rather than several lines of if.  Anything not in the list falls through to the help message.

- Expand the help message to match the message given by Doc/Makefile

- Add the "clean" target, which does the same as Makefile

- Reimplement "update" to match Makefile (call clean, call checkout.  I do wonder if maybe it should be done the other way around, though; reimplement Makfile's "update" target to simply update the svn checkouts, rather than re-pull them)

- Add separate labels for each target.  The old method of sending everything to :build was very clever and kept the file short, but limited flexibility.  It's also nice to get a message at the end (matching Makefile), which is much easier to do with separate labels.

- Add support for the PAPER environment variable in LaTeX building.

- Add coverage, doctest, pydoc-topics, dist, check, and serve targets.  "dist" cannot be completed in the same way with default tools on Windows, so all it does is build html, text, latex (a4 and letter), and epub and copy them into the dist directory.

- checkout checks for the existence of the checked out packages before checking out

- build calls checkout before building, so you can go right to "make html" instead of having to do "make checkout && make html"

- Add support for SOURCES and SPHINXOPTS environment variables.

Everything seems to work for me, but I've only been able to test on Windows 7 (32 and 64 bit).  I don't believe there's anything in there that doesn't exist in XP, though.  Also, since this version of the file has a lot of labels, this patch does somewhat rely on resolution of issue 17202.  It should probably work without any changes to .hgeol, but that can't be guaranteed.

There are a couple of drawbacks to the patched version; it's a much longer file, and it does lose a good chunk of the elegance of the original solution.  If anyone has any suggestions for regaining some of that elegance without losing any of the functionality, I am of course all ears :). In particular, I'm not a huge fan of having to have "pushd ." at the beginning of every label, but the only other solutions I came up with involved setting and unsetting a "DISALLOW_POPD" variable in several places, and a condition on the popd at :end, or forgetting the pushd/popd thing entirely.

I'm looking forward to hearing what anyone else thinks of this, positive or negative :)
msg183844 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-09 20:16
+1 I would worry more about performance than elegance.

While this is debated, can you provide a mini-patch for at least 3.3+ that corrects the bug of trying to run sphinx with python 3, so that anything but 'make checkout' fails? I would apply that immediately.
msg183848 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-03-09 20:36
Sure can.  This one should do it, at least for anyone with 3.3 installed.
msg184116 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-13 22:14
2.7 make.bat, which currently downloads sphinx v0.6.7 but not at least one needed file, does not work. See #17412. I asked there whether simply updating the sphinx version, which seems to work, is the right fix. I believe this is currently the only different in make.bat between Python versions.
msg184126 - (view) Author: Roundup Robot (python-dev) Date: 2013-03-14 01:42
New changeset 9b45873e5a68 by Terry Jan Reedy in branch '3.2':
Issue #17386 make.bat must run with Python 2 until Sphinx runs with Python 3.

New changeset e45db319e590 by Terry Jan Reedy in branch '3.3':
Merge with 3.2: Issue #17386

New changeset 99e72eae35bb by Terry Jan Reedy in branch 'default':
Merge with 3.3: Issue #17386
msg184127 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-14 01:49
I pushed the minipatch but did not add a news item as it may get superseded and will not affect anyone currently setting PYTHON.
msg184237 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-03-15 16:53
Applying the minipatch makes the big patch fail to apply, so here's an update.  Also changed in this version of the patch:

- README.txt is updated.  Not all the changes directly relate to this patch, but there were some needed updates in there anyway.  If desired, I can split those into a separate issue, but all the changes are fairly minor.

- Added *VERSION variables for each of the tools for easy overriding and easier-to-find, single place to update.

Terry, you're right about make.bat being very similar between versions.  About the only other difference I saw was the removal of the line that prints the Python version in use, and the version of Pygments used.  I'd be happy to make patches for backporting this as far as desired (assuming, of course, that it is accepted and is desirable to backport :-).
msg186036 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-04-04 14:49
Any comments?  Christian, I'd especially like to hear from you since it looks like you were the original author of Doc/make.bat (issue1472).
msg186677 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-04-12 20:27
Here's a new patch to address Ezio's review comments, including more updates to Doc/README.txt, the addition of an 'htmlview' target to make.bat, and documentation of the 'htmlview' target in the Makefile usage message.
msg186768 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-04-13 17:21
Terry, do you want to test and commit this?
Should it be backported to 3.3 (and possibly 2.7) too?
msg186785 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-04-13 18:29
I do want to test this. If my regular machine does not get fixed soon, I will install svn on my current substitute to do so.
msg187033 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-04-15 21:25
I caught a small oversight.  This new patch changes the example in README.txt to ``make html "PYTHON2=C:\Python27\python.exe"`` (PYTHON->PYTHON2).  Also, I added a bit of backward compatibility to make.bat; %PYTHON2% will default to %PYTHON% if it is set.  Otherwise, %PYTHON% is ignored and unchanged.
msg187666 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-04-23 19:07
After finally finding a way to check the status of the pushd stack, I found a bug in my patch whereby an extra directory is left on the pushd stack at the end of each run.  As such, the version 5 patch changes from using 'goto end' to using 'exit /B !ERRORLEVEL!', along with some other changes related to that change.  Also, because of that change, error handling is made a little easier, so I've changed around some of the messages and the conditions for printing them.  Everything still seems to work.

For anyone testing this patch, I would suggest using the command "prompt $+%PROMPT%" if "$+" is not already part of your prompt.  This will add a '+' character to the beginning of your prompt for every dir on the pushd stack.
msg217076 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-04-23 16:32
Having looked at this again, the current patch is just far bigger than it needs to be and tries to do too much, not to mention being rather out of date now.

So, here's a much less ambitious, much simpler patch with many fewer ways it can go wrong (but also not quite as close a match to using Makefile).  Instead of only allowing specific targets, the script now only specifies non-Sphinx-builder targets (like clean, check, help, htmlview, and serve) and assumes that any first argument it doesn't recognize is supposed to be a Sphinx builder.  "htmlhelp" is still special-cased in the "build" target.  Not listing all the Sphinx builder targets explicitly means that builders like "dirhtml", "pickle", or "json", or any builders added in future Sphinx versions will just work without our having to do anything for them.
msg217419 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-04-28 21:05
New patch, fixes a typo bug in a useless statement (by removing the statement) and a few pesky tabs that made their way into make.bat.  Also, a little better organization in the vars at the top.
msg217520 - (view) Author: Roundup Robot (python-dev) Date: 2014-04-29 14:47
New changeset c75a2282166f by Zachary Ware in branch '3.4':
Issue #17386: List the 'htmlview' target in the Doc/Makefile help output.

New changeset c378c67c4170 by Zachary Ware in branch '3.4':
Issue #17386: Update Doc/README.txt to list all targets

New changeset a172f26195f6 by Zachary Ware in branch '3.4':
Issue #17386: Expand Doc/make.bat to be much more similar to Doc/Makefile

New changeset 2b2577d79e80 by Zachary Ware in branch 'default':
Closes #17386: Merge with 3.4
Date User Action Args
2014-04-29 14:47:55python-devsetstatus: open -> closed
resolution: fixed
messages: + msg217520

stage: patch review -> resolved
2014-04-28 21:05:48zach.waresetfiles: + issue17386.v2.diff

messages: + msg217419
2014-04-23 16:32:03zach.waresetfiles: + issue17386.diff
versions: + Python 3.5
messages: + msg217076

assignee: docs@python -> zach.ware
stage: commit review -> patch review
2013-04-23 19:07:04zach.waresetfiles: + win_doc_make.v5.diff

messages: + msg187666
2013-04-15 21:25:21zach.waresetfiles: + win_doc_make.v4.diff

messages: + msg187033
2013-04-13 18:29:53terry.reedysetmessages: + msg186785
2013-04-13 17:21:41ezio.melottisetmessages: + msg186768
stage: patch review -> commit review
2013-04-12 20:27:51zach.waresetfiles: + win_doc_make.v3.diff

messages: + msg186677
2013-04-04 14:49:56zach.waresetmessages: + msg186036
2013-03-15 16:58:59zach.waresetfiles: + win_doc_make.v2.diff
2013-03-15 16:58:42zach.waresetfiles: - win_doc_make.v2.diff
2013-03-15 16:53:51zach.waresetfiles: + win_doc_make.v2.diff

messages: + msg184237
2013-03-14 01:49:30terry.reedysetmessages: + msg184127
2013-03-14 01:42:08python-devsetnosy: + python-dev
messages: + msg184126
2013-03-13 22:14:25terry.reedysetmessages: + msg184116
2013-03-09 20:36:12zach.waresetfiles: + win_doc_make_minipatch.diff

messages: + msg183848
2013-03-09 20:16:12terry.reedysetnosy: + terry.reedy, tim.golden, brian.curtin
messages: + msg183844
2013-03-09 19:27:39ezio.melottisetnosy: + ezio.melotti

stage: patch review
2013-03-08 22:33:14zach.warecreate