When building Python (cpython repository, tip revision on the default branch) using Visual Studio 2015 (version downloaded 12/02/2015) I get a message in the build:

C:\Work\Projects\cpython\PCbuild\_freeze_importlib.vcxproj(98,5): error : importlib.h has been updated. You will need to rebuild  pythoncore to see the changes.

This is using the command "PCBuild\build.bat -p x64".

When I check hg status after the build, Python\importlib.h shows as modified.

(I have 2 changed files in the repo, one a documentation file and the other launcher.c. Neither change should affect importlib).

This implies to me that someone didn't include importlib.h in their commit when they changed something that required it to be rebuilt.  However, if it's a Windows-only thing then perhaps there is something else afoot.  Steve?

What does the diff to importlib.h look like?  Does it look like a real change to importlib or is it something innocuous like a change in whitespace or the path separator in the heading?

Windows is probably the only platform that detects when _freeze_importlib makes a change and aborts the build :)

I'd be interested to see what the diff is though. My guess is some of the recent marshal changes may have affected the output.

Ah. It's 4K lines of everything changing. Looks like it might be an EOL issue. The build seems to be changing it from LF line endings to CRLF line endings.

Ah ha.  Do you have the hg eol extension enabled?  If not, you
probably should; it makes for far fewer headaches on Windows, and
should make this issue go away.

We should probably try to mitigate the problem anyway; Steve, how hard
would it be to change the 'see if the newly generated file actually
changed anything' logic to check line endings?

It's possible, but the eol extension is the best way to handle it.

Doing the comparison is relatively easy (we can just strip all '\r' and '\n' characters and compare), but if it has changed, it's much harder to update the new file to match whatever the old file had. With eol, it'll always be \r\n and it Just Works.

Leave this on me to update the check. If someone does try and push a change with EOL changes it'll be blocked anyway.

I'm just about finished for the night but I'll try the eol extension tomorrow. I don't really use it much, though, my practice is generally to make all my tools use LF. I certainly didn't do anything that would change that file, so something in the build must be writing it in CRLF line ending mode.

The other option would be to change _freeze_importlib to force it to
use \n instead of \r\n; I'm not sure which change would be harder to
do.

Paul: fair enough; I've just found that keeping eol enabled reduces
headaches due to line endings to almost nothing, just because I never
have to think about them at all (except for the rare files that are
marked as 'BIN' in .hgeol).

I noticed a few days ago that this file had changed as I got a merge conflict using the tortoisehg sychronize command.  At the time I didn't think much of it and believe I simply let hg sort it out.  The changeset was 8ab6b404248c #21295.

No, enabling the eol extension didn't fix it :-(

Correction. Looks like if I enable the eol extension, then hg revert importlib.h, then rebuild, the problem then goes away.

So basically you need the eol extension enabled *before* you update/clone your checkout.

Paul: wrt. your statement "eol extension tomorrow. I don't really use it much,". This can't be really true: either you use it, or you don't. If you use it at all, then you also use it MUCH: Python is a heavy user of the eol extension, and the entire Python hg repository can't work without it. It's an absolute requirement that anybody working on the CPython repository has the eol extension enabled.

In fact, the eol extension was added to Mercurial because of CPython, and was basically designed on python-dev. Look at CPython's .hgeol file to see what files are affected by it (it's basically every file under source control).

Martin, thanks for the clarification. When I said "I don't use it much" what I meant was that I've never had it enabled, and for all of the repositories I use, not doing so has never been a problem (until now). In contrast, git's autocrlf feature, which is enabled by default, *has* caused me issues, so I tend to work on the basis that I'm better making sure my tools interoperate properly with Unix-style line endings than making my VCS translate for me.

I was reluctant to enable the eol extension globally because of this. I've just discovered however that extensions can be enabled on a per-repo basis, which is a nice compromise, or I may just enable it globally and learn how to use it properly for other projects.

I recall the discussion when the eol extension was created. However, I had mistakenly got the impression that the extension was for people who needed to work with CRLF extensions locally (IIRC, Visual Studio was a tool that preferred CRLF). Clearly that was wrong, and I'll switch to using the eol extension now.

(As an aside, one of the downsides of having lurked on the edges of the python-dev community for as long as I have is that you think you know the processes, when actually they've changed since you last checked. Lesson learned - I'll reread the docs next time :-))

New changeset f9a43e2a3877 by Steve Dower in branch 'default':
Issue #23461: Normalise line endings when comparing old and new contents of importlib.h
https://hg.python.org/cpython/rev/f9a43e2a3877

The changeset will skip updating importlib.h if the only change is line endings, but if there are other changes it will be updated to a CRLF file. This will be fine if the eol extension is enabled, and h.p.o will reject pushes that have CRLF line endings.

However, this will make it much easier for non-core devs to build from source, whether from h.p.o or the tarballs.

Thanks, Steve!