This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Disable code coverage
Type: Stage: resolved
Components: Versions:
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: ammar2, brett.cannon, larry, rhettinger, skrah, terry.reedy, vstinner, xtreak
Priority: normal Keywords: patch

Created on 2020-02-20 23:38 by skrah, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 18680 merged ammar2, 2020-02-27 21:32
Messages (20)
msg362367 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-02-20 23:38
The automated code coverage on GitHub is quite inaccurate and needlessly flags PRs as red.

I'd prefer to make this opt-in.
msg362372 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2020-02-21 00:17
Agreed, it's way too noisy. This PR which touches absolutely no code

claims to "increase coverage by 1.01%."

This doesn't really add much value and only adds noise in the pull requests.
msg362384 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-02-21 07:37
I concur.
msg362385 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2020-02-21 07:45
Just to clarify is it about just disabling the automatic comment about code coverage on PRs or the code coverage build itself?
msg362387 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-02-21 08:34
I'd definitely disable the automatic comment and prefer that the build
happens on rather than affecting the GitHub build
msg362388 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2020-02-21 08:42
Thanks for the clarification, I agree on disabling automatic coverage comments. Aren't these builds already optional in Travis marked as allow failures and status is reported once the required builds pass though the coverage builds keep running?
msg362409 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-02-21 15:39
They are allowed failures but the build is still marked in red:

So if you look at the front page you have to click through red results
only to find that the reason is code coverage.
msg362412 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2020-02-21 16:16
Also just to clarify, the actual coverage build which measures the build. That is:

is fine. The build succeeds and the coverage can be seen online at to get a high-level overview of coverage.

The integration with codecov as a status check and it's comments are the real nuisance.
msg362450 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-02-22 06:34
The recently reinstated long 30+ line reports are useless nuisance for me.  I delete them.  A single line line to a report, without flagging, would be OK.
msg362512 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-02-23 14:19
For me even a mail with a single line would be too much. I can filter that in my mail client but not on GitHub.

Speaking about that, I also don't want to get mail from Bevedere stating that I, in fact, have signed a CLA any time I open a PR.
msg362838 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-02-27 21:08
The codecov config lives at and the docs for the file are at Specifically, the comment feature is covered at and the status check is at

Nothing has changed in the config, though, so it might be a change related to and thus the file path to the config might need to be added to the upload step in the following files:


(I would submit a PR myself but I'm still digging myself out from under email after vacation.)
msg362841 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2020-02-27 21:19
Thanks for the pointer Brett, I'll submit a PR.
msg362848 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-02-27 23:08
New changeset 766b7546a564c8e386a3c31eb06fc1b55e8f5a25 by Ammar Askar in branch 'master':
bpo-39704: Explicitly pass the path to codecov config (GH-18680)
msg362850 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2020-02-28 00:22
Can someone with access check that has the right config? The latest run after my PR was pushed through still has the status check running :(
msg362863 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2020-02-28 03:50
Looks like it was just cached, the latest pull request didn't get a codecov comment nor was it ran on the latest commit:

Should this be back-ported so backport pull requests/pull requests to other versioned branches don't get affected by this either?
msg362904 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-02-28 18:55
I don't know if backporting will be needed; probably depends on the CI and whether they always pull from master or the branch that was affected. But I just tried backporting regardless and there's conflicts, so it will have to be done manually.
msg363191 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2020-03-02 16:49
Just a quick update, I think this is a codecov bug as per here:

The yaml configuration doesn't show up here:

While we wait for a response from codecov, we can fix this in the interim by overriding the settings at an organization level. An owner of the Python organization can enable this here: the organization level stuff would be here:


comment: off
    changes: off
    project: off
    patch: off

as the config should hopefully fix this for now.
msg363259 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-03 13:38
Ernest updated the organization configuration to:
comment: off
    changes: off
    project: off
    patch: off

which gives:

    "comment": false, 
    "coverage": {
        "status": {
            "changes": false, 
            "patch": false, 
            "project": false

Codecov should no longer add comments to pull requests. I close the issue. Reopen it if Codecov adds *new* comments to pull requests.

Thanks Ernest!
msg365595 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-02 13:37
FYI I created bpo-40156: "codecov/patch stills runs on 3.5 and 3.6 branches".
msg365628 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2020-04-02 21:32
Since explicit is better than implicit: yes, we do need backports.  PRs against 3.5 are getting marked red because of automated codecov complaints.
Date User Action Args
2022-04-11 14:59:26adminsetgithub: 83885
2020-04-02 21:32:34larrysetnosy: + larry
messages: + msg365628
2020-04-02 17:53:08gvanrossumsetnosy: - gvanrossum
2020-04-02 13:37:48vstinnersetmessages: + msg365595
2020-03-03 13:38:24vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg363259

stage: patch review -> resolved
2020-03-02 16:49:33ammar2setmessages: + msg363191
2020-03-02 16:43:40gvanrossumsetnosy: + gvanrossum
2020-03-02 16:33:59vstinnersetnosy: + vstinner
2020-02-28 18:55:13brett.cannonsetmessages: + msg362904
2020-02-28 03:50:09ammar2setmessages: + msg362863
2020-02-28 00:22:20ammar2setmessages: + msg362850
2020-02-27 23:08:37brett.cannonsetmessages: + msg362848
2020-02-27 21:32:33ammar2setkeywords: + patch
stage: patch review
pull_requests: + pull_request18039
2020-02-27 21:19:04ammar2setmessages: + msg362841
2020-02-27 21:08:07brett.cannonsetnosy: + brett.cannon
messages: + msg362838
2020-02-23 14:19:48skrahsetmessages: + msg362512
2020-02-22 06:34:58terry.reedysetnosy: + terry.reedy
messages: + msg362450
2020-02-21 16:16:33ammar2setmessages: + msg362412
2020-02-21 15:39:00skrahsetmessages: + msg362409
2020-02-21 08:42:40xtreaksetmessages: + msg362388
2020-02-21 08:34:42skrahsetmessages: + msg362387
2020-02-21 07:45:50xtreaksetnosy: + xtreak
messages: + msg362385
2020-02-21 07:37:28rhettingersetnosy: + rhettinger
messages: + msg362384
2020-02-21 00:17:34ammar2setnosy: + ammar2
messages: + msg362372
2020-02-20 23:38:42skrahcreate