classification
Title: Add .bat line to .hgeol
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, brian.curtin, djc, ezio.melotti, georg.brandl, loewis, pitrou, python-dev, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2013-02-14 06:41 by zach.ware, last changed 2015-04-13 17:15 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
batch_eol_fix.diff zach.ware, 2013-02-14 06:41 review
batch_eol_fix.diff.2 tim.golden, 2013-10-24 11:57 review
Messages (10)
msg182077 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-02-14 06:41
Most times (though not all, for some reason) I try to use Doc\make.bat update, it fails with "The system cannot find the batch label specified - update" even though it is obviously there.  The reason for this appears to be the fact that the file uses UNIX line endings[1].  As a fix, I think it would make the most sense to add a "**.bat = CRLF" line to .hgeol.

The attached patch makes that change and includes the changes that that change entails.  The complete list of affected files is as follows:

Doc\make.bat
Lib\ctypes\macholib\fetch_macholib.bat
Lib\idlelib\idle.bat
Modules\_decimal\tests\runall.bat
PCbuild\build.bat
PCbuild\build_env.bat
PCbuild\build_pgo.bat
PCbuild\build_ssl.bat
PCbuild\env.bat
PCbuild\idle.bat
PCbuild\rt.bat
Tools\buildbot\build-amd64.bat
Tools\buildbot\build.bat
Tools\buildbot\buildmsi.bat
Tools\buildbot\clean-amd64.bat
Tools\buildbot\clean.bat
Tools\buildbot\external-amd64.bat
Tools\buildbot\external-common.bat
Tools\buildbot\external.bat
Tools\buildbot\test-amd64.bat
Tools\buildbot\test.bat
Tools\unicode\genwincodecs.bat

As far as I know, there is no reason to believe that the changes to line endings in these files would cause any change to behavior other than possibly avoid strange and random bugs in their execution, particularly in Doc\make.bat.

If we don't want to a blanket rule like this, I think we should at least add a rule for Doc\make.bat to convert it to CRLF line endings to avoid the bug with :update.

Thanks,

Zach

[1] http://stackoverflow.com/questions/232651/why-the-system-cannot-find-the-batch-label-specified-is-thrown-even-if-label-e/232674#232674
msg182182 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-02-15 22:07
**.bat used to be there, but it was removed in 1762d79eab65.

I have a very faint memory of some discussion about bat files and EOLs, but the best I could find was:
http://mail.python.org/pipermail/python-dev/2012-May/119220.html
http://mail.python.org/pipermail/python-dev/2012-May/119225.html

This seems to suggest that CRLF shouldn't be necessary.

There's also an older discussion about adding CRLF for make.bat here (the discussion is actually somewhat unrelated, but back then adding these entries sounded like a good idea): http://mail.python.org/pipermail/python-committers/2011-May/001685.html
msg182187 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-02-15 22:41
> This seems to suggest that CRLF shouldn't be necessary.

It's not necessary on .bat files without labels, but those with labels are subject to the (Windows) bug described in my first message and link.

Adding an entry specifically for Doc/make.bat and any other batch scripts with labels would be fine as well.
msg192356 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-07-05 20:34
I have recently learned why this issue had seemed transient to me previously: the hg eol extension apparently causes hg to create all files that are not explicitly set to LF or BIN using CRLF on Windows.  Thus, this is what happens:

"""
P:\Python\cpython\Doc>rem In a fresh checkout made with the 'eol' extension disabled:

P:\Python\cpython\Doc>make.bat update
The system cannot find the batch label specified - update

P:\Python\cpython\Doc>cd ..

P:\Python\cpython>hg up null
0 files updated, 0 files merged, 3689 files removed, 0 files unresolved

P:\Python\cpython>hg --config extensions.eol= up default
3689 files updated, 0 files merged, 0 files removed, 0 files unresolved

P:\Python\cpython>cd Doc

P:\Python\cpython\Doc>make.bat update
Updating 'tools\sphinx':
At revision 89001.
Updating 'tools\docutils':
At revision 89001.
Updating 'tools\jinja2':
At revision 89001.
Updating 'tools\pygments':
At revision 89001.

P:\Python\cpython\Doc>
"""

Thus, a usable workaround is to enable the eol extension in hgrc and make your checkout that way.  However, this doesn't help anyone who is trying to build from a tarball; Doc/make.bat will have LF line endings and at least 'update' will not work.

Perhaps a less extensive change would be to add "**.bat  = BIN", and then convert Doc/make.bat to CRLF explicitly?
msg193085 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-07-15 10:05
Zachary: What Windows version do you use? Glen Linderman claims that at least Windows 7 should work fine with cmd files containing only LF, and suggested that this may also be the case on Windows XP.

Are you using a non-standard command interpreter by any chance?
msg193099 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-07-15 13:46
I'm on 32 bit Windows 7 Pro, SP1, and have also reproduced the issue on 64 bit Windows 7 Home Premium, SP1. I use the standard Command Prompt (cmd.exe) on both.
msg201116 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-10-24 11:57
I'm running Win7 and have the same problem, with a fresh checkout and the eol extension disabled. The attached (updated) patch does solve the issue. I can't see any reason not to apply it, even if it's not needed in many cases.
msg206651 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-12-19 21:40
Any objections to proceeding with this?
msg233357 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015-01-03 14:40
No objections so proceeding is in order here I take it?
msg240665 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-13 17:15
New changeset 019c17a4198a by Zachary Ware in branch '2.7':
Issue #17202: Add .bat to .hgeol to force them to CRLF.
https://hg.python.org/cpython/rev/019c17a4198a

New changeset 640ccb924b5f by Zachary Ware in branch '3.4':
Issue #17202: Add .bat to .hgeol to force them to CRLF.
https://hg.python.org/cpython/rev/640ccb924b5f

New changeset 9b2ea49213fd by Zachary Ware in branch 'default':
Closes #17202: Merge with 3.4
https://hg.python.org/cpython/rev/9b2ea49213fd
History
Date User Action Args
2015-04-13 17:15:22python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg240665

resolution: fixed
stage: patch review -> resolved
2015-01-03 14:40:53BreamoreBoysetnosy: + BreamoreBoy
messages: + msg233357
2013-12-19 21:40:04zach.waresetmessages: + msg206651
2013-10-24 11:57:10tim.goldensetfiles: + batch_eol_fix.diff.2

messages: + msg201116
2013-07-15 13:46:08zach.waresetmessages: + msg193099
2013-07-15 10:05:03loewissetmessages: + msg193085
2013-07-05 20:34:57zach.waresetmessages: + msg192356
2013-02-15 22:41:34zach.waresetmessages: + msg182187
2013-02-15 22:07:35ezio.melottisetmessages: + msg182182
2013-02-15 21:59:06djcsetnosy: + georg.brandl, pitrou
2013-02-15 21:43:56ezio.melottisetnosy: + loewis, djc, ezio.melotti

type: behavior
stage: patch review
2013-02-14 06:41:26zach.warecreate