classification
Title: Windows buildbots broken by the migration to GitHub (meta issue)
Type: Stage: resolved
Components: Tests, Windows Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: jkloth, paul.moore, r.david.murray, serhiy.storchaka, steve.dower, tim.golden, vstinner, zach.ware
Priority: critical Keywords: buildbot

Created on 2017-02-11 01:58 by vstinner, last changed 2017-06-28 00:57 by vstinner. This issue is now closed.

Messages (9)
msg287587 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-11 01:58
(*) I noticed test_sax and test_random failures on Windows since CPython moved to GitHub. These failures seem to be caused by newlines: see issue #27425.


(*) "AMD64 Windows7 SP1 3.x" fails on the "git" step with:

   'git' is not recognized as an internal or external command

I contacted the slave owner, Jeremy, to ask him to install git.
msg287603 - (view) Author: Jeremy Kloth (jkloth) * Date: 2017-02-11 15:27
After ensuring that Git is available to the buildbot service, the initial update fails due to the existing build directory containing the Mercurial checkout.

After manually removing the contents of the build directories, the update task successfully completes.
msg287608 - (view) Author: Jeremy Kloth (jkloth) * Date: 2017-02-11 16:05
Tests are now failing due to end-of-line (?) differences.

I did a quick reading of the devguide, but didn't see any mention of the settings to be used for Git on Windows wrt core.autocrlf.
msg287610 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-02-11 16:07
We really need to be able to function regardless of the core.autocrlf setting.  See #27425 for more about the EOL issues.
msg287613 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-02-11 17:20
My vote would be to make the tests not presume anything about line endings, to the point of converting test files on the fly to be sure.

I'm mostly free today (in a different city with nothing to do), so if my hotel internet will let me SSH to github then I can work on this today. If people are around, I'll be on IRC too.
msg287621 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-02-11 19:55
Some of the tests (notably email tests) are *testing* platform line endings.  In tests that I rewrote I did my best to make the tests independent of the line endings of checked in files (that is, constructing the files-to-test in the test itself instead of checking them in), but I wouldn't be surprised if there were a few tests that weren't converted to that paradigm.  (I haven't looked to see if there are any test.test_email.test_email failures, which is where they'd be.)
msg287623 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-11 20:03
Does Git support per-file settings like the svn:eol-style property in Subversion or .hgeol in Mercurial?
msg287625 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-02-11 20:37
> Does Git support per-file settings like the svn:eol-style property in Subversion or .hgeol in Mercurial?

The .gitattributes file seems to be full of glob patterns, so I assume a full file path will work.

However, that doesn't help with tarballs. Fixing the tests is the best way.
msg297091 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-06-28 00:57
Windows buildbots are green since at least 2 months, so I close this issue.

There is one remaining issue, but specific to the Windows *installer*: see bpo-30716 and bpo-27425.
History
Date User Action Args
2017-06-28 00:57:41vstinnersetstatus: open -> closed
messages: + msg297091

dependencies: - Tests fail because of git's newline preferences on Windows
resolution: fixed
stage: resolved
2017-02-11 20:37:41steve.dowersetmessages: + msg287625
2017-02-11 20:03:59serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg287623
2017-02-11 19:55:25r.david.murraysetnosy: + r.david.murray
messages: + msg287621
2017-02-11 17:20:39steve.dowersetmessages: + msg287613
2017-02-11 16:07:35zach.waresetdependencies: + Tests fail because of git's newline preferences on Windows
messages: + msg287610
2017-02-11 16:06:10zach.waresetpriority: normal -> critical
nosy: + paul.moore, tim.golden, zach.ware, steve.dower

components: + Windows
versions: + Python 2.7, Python 3.5, Python 3.6
2017-02-11 16:05:28jklothsetmessages: + msg287608
2017-02-11 15:27:41jklothsetnosy: + jkloth
messages: + msg287603
2017-02-11 01:58:36vstinnercreate