classification
Title: Make Azure Pipelines optional on GitHub PRs
Type: Stage: resolved
Components: Tests Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, miss-islington, ned.deily, python-dev, shihai1991, steve.dower, vstinner
Priority: normal Keywords: patch

Created on 2020-03-03 17:05 by vstinner, last changed 2020-05-15 15:12 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
Annotation 2020-03-05 223851.png steve.dower, 2020-03-05 22:42
Pull Requests
URL Status Linked Edit
PR 18769 closed vstinner, 2020-03-03 17:10
PR 18818 merged steve.dower, 2020-03-06 22:44
PR 18820 merged miss-islington, 2020-03-07 00:12
PR 18821 merged miss-islington, 2020-03-07 00:12
PR 18835 closed python-dev, 2020-03-08 03:05
Messages (50)
msg363279 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-03 17:05
The Azure Pipelines jobs have been reimplemented as GitHub actions which are better integrated with GitHub:

* Docs / Docs (pull_request)
* Tests / Windows (x64) (pull_request)
* Tests / macOS (pull_request)
* Tests / Ubuntu (pull_request)
* etc.

Azure Pipelines runs the same jobs, but it looks slower. It is voting and so prevents to merge a PR until it completes.

I propose to simply remove the job.

I already proposed it on python-dev:
https://mail.python.org/archives/list/python-dev@python.org/message/NC2ZS4WSF5AYGUUMBB7I4YIQ4YJXWBA5/

In this thread:
https://mail.python.org/archives/list/python-dev@python.org/thread/2NSPJUEWULTLLALR3HY3H2PRYAUT474C/#NC2ZS4WSF5AYGUUMBB7I4YIQ4YJXWBA5
msg363289 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-03 18:29
Deleting the files is not the right first step.

First, it needs to be changed to a non-required check.

Then, I can use the web UI to disable it starting.

*Then*, we can remove *some* of the files in the directory. Others are used for the official release, and have to stay.

PR 18769 should *not* be merged.
msg363291 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-03 18:34
> PR 18769 should *not* be merged.

Ok, I closed it.

> First, it needs to be changed to a non-required check.

Who is allowed to do that?
msg363292 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-03 18:38
> Who is allowed to do that?

Brett did it the first time. I'm having too much trouble with GitHub right now to find the current admins.
msg363382 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-03-04 19:28
Yes, I can do it.

And to answer Victor's question on the PR that he closed, we can make any individual status check required. So probably:

- Docs
- Ubuntu
- Windows x86
- Windows x64

Just let me know when we are ready to merge a PR and I will switch off Azure Pipelines and make these checks required.
msg363384 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-04 19:48
There's no PR required. We need to change the required check so when I disable new builds from running it won't break existing PRs.

As for actually changing the files in the repo, I don't see any hurry there. We can clean up a few of them, and maybe I can move the release build scripts into the PC folder (though would have to backport that through to 3.7).

But I'm ready to disable the builds from running as soon as they're not a required check.
msg363394 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-04 22:15
> And to answer Victor's question on the PR that he closed, we can make any individual status check required.

Great! I saw a macOS failure this week which was caused by an internal CI error. I hope that it will be fixed soon. In the meanwhile, I would suggest to not make the macOS job mandatory, but pay attention manually to its status ;-) Apart of macOS, other jobs look reliable.
msg363405 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-03-04 23:56
Actually, I just realized we can't make these status checks required because they don't always run. :) Our Actions are smart enough to not run when they aren't necessary, i.e. doc changes don't run the rest of the checks. And so making the OS-specific tests required would block doc tests.

Basically we would either have to waste runs on things that aren't really necessary but then be able to require runs, or we have to just rely on people paying attention to failures.

I'm personally for the latter.
msg363425 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-05 10:26
Are you sure the required check isn't just for failures? Surely GitHub is smarter than requiring checks that it can tell aren't required (as it's their logic to include/exclude, not ours).

If they're not, I now have many internal contacts there, so we can probably get it fixed :)
msg363447 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-03-05 18:40
It was an old issue that if required checks didn't run it would block, but hopefully it's fixed. :)

I have gone ahead and removed the Azure Pipelines requirement from 3.7, 3.8, and master and flipped on the check requirements for the ones I listed.
msg363455 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-05 21:24
Looks like it isn't fixed... https://github.com/python/cpython/pull/18774
msg363458 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-05 22:21
Instead of not running the job, is it technically possible to modify the jobs to do nothing for docs only changes?

.travis.yml works like that:

before_install:
  - set -e
  - |
      # Check short-circuit conditions
      if [[ "${TESTING}" != "docs" && "${TESTING}" != "doctest" ]]
      then
        if [[ "$TRAVIS_PULL_REQUEST" == "false" ]]
        then
          echo "Not a PR, doing full build."
        else
          # Pull requests are slightly complicated because $TRAVIS_COMMIT_RANGE
          # may include more changes than desired if the history is convoluted.
          # Instead, explicitly fetch the base branch and compare against the
          # merge-base commit.
          git fetch -q origin +refs/heads/$TRAVIS_BRANCH
          changes=$(git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD))
          echo "Files changed:"
          echo "$changes"
          if ! echo "$changes" | grep -qvE '(\.rst$)|(^Doc)|(^Misc)'
          then
            echo "Only docs were updated, stopping build process."
            exit
          fi
        fi
      fi
msg363460 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-05 22:42
Adding a screenshot here so I can point people at it.

Let's not rush into complicating the build steps yet - AP is basically fine. We should switch back the required checks (@Brett?)

If anything, let's add a "condition: false" to the macOS build to disable it and rely on the non-required GH check for now.
msg363466 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-03-06 00:45
I've turned off the required checks for GH Actions and flipped Azure Pipelines back on.

And to answer Victor's question, yes, you can make things conditional at the workflow, job, and job step level. I don't know what would happen if the check was moved from workflow to job level.
msg363498 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-06 09:10
It would make the job definition significantly more complicated, and I don't want to do that just to work around an issue with github until we've got positive confirmation that the behaviour is intentional and won't change.
msg363568 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-07 00:11
New changeset 31350f9af09dcff7cf6ff4b0a0a7ea595942372e by Steve Dower in branch 'master':
bpo-39837: Disable macOS tests on Azure Pipelines (GH-18818)
https://github.com/python/cpython/commit/31350f9af09dcff7cf6ff4b0a0a7ea595942372e
msg363569 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-07 00:14
I've disabled macOS builds on Pipelines, so now they're essentially advisory through the GitHub Actions build.

I also pinged some contacts about the not-very-useful behaviour of required checks vs. path filters. So will see what they say.
msg363570 - (view) Author: miss-islington (miss-islington) Date: 2020-03-07 00:29
New changeset 47b7c227048f2cb019cc3ec2fef7e867f1b232f3 by Miss Islington (bot) in branch '3.7':
bpo-39837: Disable macOS tests on Azure Pipelines (GH-18818)
https://github.com/python/cpython/commit/47b7c227048f2cb019cc3ec2fef7e867f1b232f3
msg363571 - (view) Author: miss-islington (miss-islington) Date: 2020-03-07 00:30
New changeset d692d52f4aaeb6feaabb18f18d49907dd578fbeb by Miss Islington (bot) in branch '3.8':
bpo-39837: Disable macOS tests on Azure Pipelines (GH-18818)
https://github.com/python/cpython/commit/d692d52f4aaeb6feaabb18f18d49907dd578fbeb
msg363808 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-03-10 07:48
New changeset b092892f9afd37fd3355b1e91fc5835f196ea914 by Ned Deily (Miss Islington (bot)) in branch '3.7':
bpo-39837: Disable macOS tests on Azure Pipelines (GH-18818)
https://github.com/python/cpython/commit/b092892f9afd37fd3355b1e91fc5835f196ea914
msg365460 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-01 12:53
I cannot merge a PR until it completes. It re-runs jobs which are already run as GH Actions.

There is another annoying issue with Azure Pipelines. When a job fails randomly for whatever reason, a job cannot be re-run, even if I log in Microsoft Azure.

Usually, the workaround is to close/reopen a PR to re-run all CIs.

Except that for a backport PR created automatically by miss-islington bot, when I close the PR, the bot removes its branch and so the PR cannot be re-open.

Well, the second workaround is to ask the bot to create a new PR backport. That what I did.

I did that for PR 19276 of bpo-40121. It's annoying to have to use *two* workarounds.

On the other side, Travis CI is not currently required, I don't understand why.

Is it possible to make Travis CI required and make Azure Pipelines not required?
msg365496 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-04-01 16:55
> Is it possible to make Travis CI required and make Azure Pipelines not required?

Yes, but I don't want to to do that as we have had equivalent flakiness issues with Travis which is why it isn't required ATM.

The only way to prevent flaky CI from blocking anything is to simply make no CI required and trust core devs not to merge unless they are certain they know why a CI run failed (although I don't know what that does to miss-islington). Passed that is being extremely specific about what CI is considered stable enough to block on an would probably need to be down to the OS level on top of what is being tested.
msg365503 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-04-01 18:37
Or we could remove the path filter on GitHub Actions so that all checks run even for doc-only changes? Then they can be marked as required.
msg365519 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-01 22:08
> Yes, but I don't want to to do that as we have had equivalent flakiness issues with Travis which is why it isn't required ATM.

I'm not aware of Travis CI current issue. There were issues in the past, as with any CI, right ;-) Travis CI looks quite reliable these days.

Whereas the Docs and Ubuntu GH Action job of Azure failed to install dependencies :-(
msg365535 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-02 00:18
About the "Install Dependencies" issue, I created bpo-40146: "Upgrade Azure Pipelines to OpenSSL 1.1.1f".

I don't understand how it's possible that the Ubuntu job passed in the GitHub action, but failed in Azure Pipelines. Maybe there was a download cache somehow, and the cache is now outdated.
msg365561 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-02 02:45
> About the "Install Dependencies" issue, I created bpo-40146: "Upgrade Azure Pipelines to OpenSSL 1.1.1f".

I fixed this issue.

There is another issue: sometimes, the Ubuntu VM fails to download .deb from azure.archive.ubuntu.com.

Example:
https://github.com/python/cpython/pull/19282/checks?check_run_id=553744077

---
Err:1 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 lcov all 1.13-3
  Could not connect to azure.archive.ubuntu.com:80 (52.177.174.250), connection timed out
---

This error is followed by many similar errors.
msg365604 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-04-02 16:34
> I don't understand how it's possible that the Ubuntu job passed in the GitHub action, but failed in Azure Pipelines.

Random network issues, most likely. Those are about the only flakiness I've seen in recent times.

The OpenSSL cache should only be for the from-source build we do on Ubuntu for CI. It uses the multissl script in our repo and downloads directly from openssl.org. The dependencies obtained using apt are not cached by us.
msg365609 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-04-02 17:49
> I'm not aware of Travis CI current issue. There were issues in the past, as with any CI, right ;-) Travis CI looks quite reliable these days.

That's what everyone said when Travis was required and before it went flaky the last time. ;)

The point is I don't want to keep flipping on and off required checks based on whatever CI people deem flaky or not at any one time.
msg365799 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-04 22:20
> There is another issue: sometimes, the Ubuntu VM fails to download .deb from azure.archive.ubuntu.com.

I created bpo-40188: "Azure Pipelines jobs failing randomly with: Unable to connect to azure.archive.ubuntu.com".
msg367578 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-28 23:49
Another Azure Pipeline failure on my https://github.com/python/cpython/pull/19769 PR, it looks like a random networking failure.

Sadly, I had to close/reopen my PR since there is no button to only restart the failure job, or even restart all Azure Pipeline jobs. This retrigger all CI jobs :-(

https://dev.azure.com/Python/cpython/_build/results?buildId=61891&view=logs&j=c83831cd-3752-5cc7-2f01-8276919eb334&t=6bd3fafc-d115-560e-4a08-fa9326c4b5c7

The win64 job of Azure Pipelines PR fails to build Python because it failed to fetch bzip2:

Fetching bzip2-1.0.6...
(...)
  File "C:\hostedtoolcache\windows\Python\3.8.2\x64\lib\http\client.py", line 915, in connect
    self.sock = self._create_connection(
  File "C:\hostedtoolcache\windows\Python\3.8.2\x64\lib\socket.py", line 787, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "C:\hostedtoolcache\windows\Python\3.8.2\x64\lib\socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
(...)
  File "C:\hostedtoolcache\windows\Python\3.8.2\x64\lib\urllib\request.py", line 1362, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "C:\hostedtoolcache\windows\Python\3.8.2\x64\lib\urllib\request.py", line 1322, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
(...)
c1 : fatal error C1083: Cannot open source file: 'd:\a\1\b\externals\bzip2-1.0.6\blocksort.c': No such file or directory [D:\a\1\s\PCbuild\_bz2.vcxproj]
c1 : fatal error C1083: Cannot open source file: 'd:\a\1\b\externals\bzip2-1.0.6\bzlib.c': No such file or directory [D:\a\1\s\PCbuild\_bz2.vcxproj]
c1 : fatal error C1083: Cannot open source file: 'd:\a\1\b\externals\bzip2-1.0.6\compress.c': No such file or directory [D:\a\1\s\PCbuild\_bz2.vcxproj]
c1 : fatal error C1083: Cannot open source file: 'd:\a\1\b\externals\bzip2-1.0.6\crctable.c': No such file or directory [D:\a\1\s\PCbuild\_bz2.vcxproj]
c1 : fatal error C1083: Cannot open source file: 'd:\a\1\b\externals\bzip2-1.0.6\decompress.c': No such file or directory [D:\a\1\s\PCbuild\_bz2.vcxproj]
c1 : fatal error C1083: Cannot open source file: 'd:\a\1\b\externals\bzip2-1.0.6\huffman.c': No such file or directory [D:\a\1\s\PCbuild\_bz2.vcxproj]
c1 : fatal error C1083: Cannot open source file: 'd:\a\1\b\externals\bzip2-1.0.6\randtable.c': No such file or directory [D:\a\1\s\PCbuild\_bz2.vcxproj]
msg367593 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-29 00:27
> Sadly, I had to close/reopen my PR since there is no button to only restart the failure job, or even restart all Azure Pipeline jobs. This retrigger all CI jobs :-(

Sadly (again), closing/reopening a PR re-runs all CIs. At the first run, GH Action macOS job passed. At the second run, the "Tests" step of GH Action macOS job, but I'm clueless with its logs:

https://github.com/python/cpython/pull/19776/checks?check_run_id=627916923

2020-04-28T23:33:03.5559341Z ##[section]Starting: Request a runner to run this job
2020-04-28T23:33:04.1879823Z Requesting a hosted runner in current repository's account/organization with labels: 'macos-latest', require runner match: True
2020-04-28T23:33:04.2799035Z Labels matched hosted runners has been found, waiting for one of them get assigned for this job.
2020-04-28T23:33:04.3183702Z ##[section]Finishing: Request a runner to run this job

On the web UI, I see that 6 steps completed, only the last "Tests" step failed. But can't I see logs of other steps?

I would prefer to be able to merge a PR even when Azure Pipelines fails: make the job optional.

Hopefully, GH Action macOS job is optional and so I can merge my PR ;-)

Note: I'm not sure if it's the right place to report GH Action macOS failure, but it seems to be related to Azure Pipelines.
msg367600 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-29 00:52
> Sadly (again), closing/reopening a PR re-runs all CIs. At the first run, GH Action macOS job passed. At the second run, the "Tests" step of GH Action macOS job, but I'm clueless with its logs: (...)

Oops, I looked at two different PRs. In fact, the two CI failures are unrelated.
msg367627 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2020-04-29 05:17
> Sadly (again), closing/reopening a PR re-runs all CIs. At the first run, GH Action macOS job passed. At the second run, the "Tests" step of GH Action macOS job, but I'm clueless with its logs: (...)

Oh, I encountered the same trouble twice :(
https://github.com/python/cpython/runs/628458496
msg367659 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-29 15:21
Another issue: I still see "Azure Pipelines PR Expected — Waiting for status to be reported" 15 min after I created my PR :-/ Technically, I created the PR and then pushed a second commit to the PR. The only option is to close/reopen the PR to re-trigger *all* CIs :-/
msg367673 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-04-29 16:32
Best place to report workflow issues or to have discussions about it is https://github.com/python/core-workflow/. Otherwise there were so many posts I didn't find an explicit ask of what you wanted changed, Victor.
msg367678 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-29 16:53
> Otherwise there were so many posts I didn't find an explicit ask of what you wanted changed, Victor.

I would like to make Azure Pipelines optional on GitHub PRs.

I changed the issue title to make my request more explicit.
msg367769 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-04-30 18:36
Done. You will need to check that miss-islington doesn't solely rely on required checks passing but instead all CI checks passing, otherwise this just turned off gating for PRs when auto-merging.

And I'm going to say future requests for this sort of stuff should happen on either on the core-workflow issue tracker or on discuss.python.org for better visibility.
msg367781 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-30 20:26
> Done.

Thanks.

> You will need to check that miss-islington doesn't solely rely on required checks passing but instead all CI checks passing, otherwise this just turned off gating for PRs when auto-merging.

I have no idea how miss-islington check CIs.

> And I'm going to say future requests for this sort of stuff should happen on either on the core-workflow issue tracker or on discuss.python.org for better visibility.

I'm used to report buildbot failures on bugs.python.org. Almost all issues are Python bugs, rather than issues specific to buidbot themselves.

I'm fine with reporting Azure Pipeline issues at core-workflow.

I created https://github.com/python/core-workflow/issues/365 " Make Travis CI (and Windows x64 ?) mandatory" :-)
msg367798 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-04-30 22:49
Bugs in Python should continue to be reported here.

Requests to change the workflow should be discussed on one of the core-workflow groups (I think Discourse is the primary one now, right?). Once an action is agreed upon, it gets tracked on the core-workflow tracker. That's how we decided to turn Travis off and Azure Pipelines on in the first place.

Let's just hope that Travis has stabilised compared to when we switched away from it, and maybe they have enough capacity now to handle our busy periods.
msg367799 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-04-30 22:51
Oh, and Victor, you should probably email python-dev to let everyone know you requested this change and it's been made. Otherwise people may be surprised that it changed without any discussion or notification.

This is especially important if we have to disable all platforms other than Linux to avoid blocking PRs.
msg367800 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-30 22:59
> Let's just hope that Travis has stabilised compared to when we switched away from it, and maybe they have enough capacity now to handle our busy periods.

Can't we be more flexible depending on the stability on CIs over the last weeks? I mean making a CI optional if it becomes flaky, but also try to make a CI mandatory when it becomes stable.

In my experience, no CI is reliable and the stability varies a lot over time.

In the past, the macOS job was very reliable. I have no idea why it became so flaky, but I don't have the bandwidth to investigate, moreover it seems like some issues are internal to Azure Pipelines / GH Actions, and I don't have access to these.

I'm trying to do the best with my limited time.
msg367801 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-04-30 23:03
> In the past, the macOS job was very reliable. I have no idea why it became so flaky, but I don't have the bandwidth to investigate, moreover it seems like some issues are internal to Azure Pipelines / GH Actions, and I don't have access to these.

FWIW, I took a quick look at it and, with nothing to go on in the way of visible messages, the best guess I could come up with is that the test run step is hitting a time out and that, in that case, no status is shown. Anyone know if that is a reasonable guess?  The next question would be why are the tests taking that long on that macOS instance.
msg367802 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-30 23:05
Steve:
> Requests to change the workflow should be discussed on one of the core-workflow groups (I think Discourse is the primary one now, right?). Once an action is agreed upon, it gets tracked on the core-workflow tracker. That's how we decided to turn Travis off and Azure Pipelines on in the first place.

Ok.


> Oh, and Victor, you should probably email python-dev to let everyone know you requested this change and it's been made. Otherwise people may be surprised that it changed without any discussion or notification.

I'm not sure of what you mean by "no discussion", this issue has many comments.


> This is especially important if we have to disable all platforms other than Linux to avoid blocking PRs.

I would be more confident if we could make at least one Windows job mandatory.

I have no opinion on msg363405, so I'm fine with Brett choice ("we have to just rely on people paying attention to failures").

I don't know how to modify the Windows job to do nothing if it's a documentation change only.

macOS was already non-voting (optional), no?
msg367804 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-04-30 23:12
> FWIW, I took a quick look at it and, with nothing to go on in the way of visible messages, the best guess I could come up with is that the test run step is hitting a time out and that, in that case, no status is shown. Anyone know if that is a reasonable guess?

I think it depends on the timeout. Some of my Ubuntu builds occasionally get hard-stuck on tkinter tests, so apparently it's possible for that to spoil CI. But I believe Pipelines is going to try and terminate the process "nicely" first.
msg367805 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-04-30 23:16
> I'm not sure of what you mean by "no discussion", this issue has many comments.

Let's say, no consensus. There were three votes cast in this discussion - yours (+1), mine (-1) and Brett's (I'll assume +0 because he made the change, despite saying he didn't want to ;) ).

Meanwhile, *everyone* is impacted, some people very negatively. The rest of the dev team need to know that it was a deliberate change.

> I would be more confident if we could make at least one Windows job mandatory.

Yes, so would I :)

> I don't know how to modify the Windows job to do nothing if it's a documentation change only.

I can do it when I get time, but it's not very high on my list. I suggest looking at the Azure Pipelines definition, kind of like how I looked at the Travis definition to figure it out.

> macOS was already non-voting (optional), no?

Only because you complained about it here :) That was PR 18818
msg367806 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-30 23:35
I understood that such issue should be discussed in the Core Workflow category of Discourse, so I created:

"Make one Windows CI job mandatory"
https://discuss.python.org/t/make-one-windows-ci-job-mandatory/4047

I suggest to continue the discussion there.
msg367807 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-30 23:41
Me:
> macOS was already non-voting (optional), no?

Steve:
> Only because you complained about it here :) That was PR 18818

Alright, I forgot about the whole history. Well, it's not my fault if macOS decided to fail :-)

I did my part, I fixed os.getgrouplist() which started (!?) to fail on the macOS job of Azure (in fact, it was an old issue which wasn't noticed previously): https://bugs.python.org/issue40014

I'm not sure what to do with macOS job which never starts or fail with empty logs. I don't see what we can do on the Python side. It *seems* to be more on the Azure side which is a blackbox to me. Maybe Steve you may ask around you at Microsoft? If you feel that you can do something to unblock the situation, please open an issue.

Note: I would also prefer to have a voting macOS job, but it's not like I can fix the macOS job myself, so I let others handle this one ;-)
msg367861 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-05-01 16:35
> Can't we be more flexible depending on the stability on CIs over the last weeks?

No because I'm tired of flipping CI on and off as mandatory based on the whims of CI systems and their stability. Either people need to accept CI is flaky or everyone needs to be careful in how they merge PRs by checking failures are legit.

And that's why I flipped off Azure Pipelines: I am not changing any more branch protections until a full discussion is had somewhere and there's consensus on what should be mandatory and stay mandatory for several months barring emergencies.
msg368358 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-07 17:48
I created a follow-up issue to have again a mandatory Windows pre-commit CI: bpo-40548.
msg368952 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-15 15:12
> Oh, and Victor, you should probably email python-dev to let everyone know you requested this change and it's been made. Otherwise people may be surprised that it changed without any discussion or notification.

I wanted to wait until the situation was being clarified. I fixed the "documentation only" issue in GitHub Action workflow. I sent an email to python-committers rather than python-dev, core devs are the first concerned by workflow changes:

https://mail.python.org/archives/list/python-committers@python.org/thread/B6WVI254L7GEOCKUOHZ6XBZD4GCLAIBV/

Slowly, it seems like the situation is being resolved.
History
Date User Action Args
2020-05-15 15:12:05vstinnersetmessages: + msg368952
2020-05-07 17:48:04vstinnersetmessages: + msg368358
2020-05-01 16:35:04brett.cannonsetmessages: + msg367861
2020-04-30 23:41:59vstinnersetmessages: + msg367807
2020-04-30 23:35:56vstinnersetmessages: + msg367806
2020-04-30 23:16:05steve.dowersetmessages: + msg367805
2020-04-30 23:12:40steve.dowersetmessages: + msg367804
2020-04-30 23:05:57vstinnersetmessages: + msg367802
2020-04-30 23:03:16ned.deilysetmessages: + msg367801
2020-04-30 22:59:15vstinnersetmessages: + msg367800
2020-04-30 22:51:20steve.dowersetmessages: + msg367799
2020-04-30 22:49:55steve.dowersetmessages: + msg367798
2020-04-30 20:26:38vstinnersetmessages: + msg367781
2020-04-30 18:36:36brett.cannonsetstatus: open -> closed
resolution: fixed
messages: + msg367769

stage: patch review -> resolved
2020-04-29 16:53:57vstinnersetmessages: + msg367678
2020-04-29 16:32:05brett.cannonsetmessages: + msg367673
2020-04-29 15:21:39vstinnersettitle: Remove Azure Pipelines from GitHub PRs -> Make Azure Pipelines optional on GitHub PRs
2020-04-29 15:21:25vstinnersetmessages: + msg367659
2020-04-29 05:17:21shihai1991setnosy: + shihai1991
messages: + msg367627
2020-04-29 00:52:53vstinnersetmessages: + msg367600
2020-04-29 00:27:52vstinnersetmessages: + msg367593
2020-04-28 23:49:27vstinnersetmessages: + msg367578
2020-04-04 22:20:43vstinnersetmessages: + msg365799
2020-04-02 17:49:27brett.cannonsetmessages: + msg365609
2020-04-02 16:34:00steve.dowersetmessages: + msg365604
2020-04-02 02:45:20vstinnersetmessages: + msg365561
2020-04-02 00:18:20vstinnersetmessages: + msg365535
2020-04-01 22:08:27vstinnersetmessages: + msg365519
2020-04-01 18:37:04steve.dowersetmessages: + msg365503
2020-04-01 16:55:48brett.cannonsetmessages: + msg365496
2020-04-01 12:53:13vstinnersetmessages: + msg365460
2020-03-10 07:48:52ned.deilysetnosy: + ned.deily
messages: + msg363808
2020-03-08 03:05:16python-devsetnosy: + python-dev
pull_requests: + pull_request18192
2020-03-07 00:30:13miss-islingtonsetmessages: + msg363571
2020-03-07 00:29:55miss-islingtonsetmessages: + msg363570
2020-03-07 00:14:10steve.dowersetmessages: + msg363569
versions: + Python 3.7, Python 3.8
2020-03-07 00:12:09miss-islingtonsetpull_requests: + pull_request18179
2020-03-07 00:12:00miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request18178
2020-03-07 00:11:50steve.dowersetmessages: + msg363568
2020-03-06 22:44:58steve.dowersetpull_requests: + pull_request18176
2020-03-06 09:10:36steve.dowersetmessages: + msg363498
2020-03-06 00:45:00brett.cannonsetmessages: + msg363466
2020-03-05 22:42:08steve.dowersetfiles: + Annotation 2020-03-05 223851.png

messages: + msg363460
2020-03-05 22:21:40vstinnersetmessages: + msg363458
2020-03-05 21:24:45steve.dowersetmessages: + msg363455
2020-03-05 18:40:44brett.cannonsetmessages: + msg363447
2020-03-05 10:26:15steve.dowersetmessages: + msg363425
2020-03-04 23:56:31brett.cannonsetmessages: + msg363405
2020-03-04 22:15:45vstinnersetmessages: + msg363394
2020-03-04 19:48:30steve.dowersetmessages: + msg363384
2020-03-04 19:28:51brett.cannonsetmessages: + msg363382
2020-03-03 18:38:52steve.dowersetnosy: + brett.cannon
messages: + msg363292
2020-03-03 18:34:27vstinnersetmessages: + msg363291
2020-03-03 18:29:16steve.dowersetmessages: + msg363289
2020-03-03 18:03:51brett.cannonsetnosy: + steve.dower
2020-03-03 17:10:22vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request18128
2020-03-03 17:05:19vstinnercreate