The automated code coverage on GitHub is quite inaccurate and needlessly flags PRs as red.

I'd prefer to make this opt-in.

Agreed, it's way too noisy. This PR which touches absolutely no code https://github.com/python/cpython/pull/18583#issuecomment-589432937

claims to "increase coverage by 1.01%."

This doesn't really add much value and only adds noise in the pull requests.

I concur.

Just to clarify is it about just disabling the automatic comment about code coverage on PRs or the code coverage build itself?

I'd definitely disable the automatic comment and prefer that the build
happens on buildbot.python.org rather than affecting the GitHub build
status.

Thanks for the clarification, I agree on disabling automatic coverage comments. Aren't these builds already optional in Travis marked as allow failures and status is reported once the required builds pass though the coverage builds keep running?

They are allowed failures but the build is still marked in red:

https://github.com/python/cpython/pull/18567


So if you look at the front page you have to click through red results
only to find that the reason is code coverage.

Also just to clarify, the actual coverage build which measures the build. That is:

https://github.com/python/cpython/blob/d4d17fd2cf69e7c8f4cd03fbf2d575370945b952/.travis.yml#L75-L114

is fine. The build succeeds and the coverage can be seen online at https://codecov.io/gh/python/cpython to get a high-level overview of coverage.

The integration with codecov as a status check and it's comments are the real nuisance.

The recently reinstated long 30+ line reports are useless nuisance for me.  I delete them.  A single line line to a report, without flagging, would be OK.

For me even a mail with a single line would be too much. I can filter that in my mail client but not on GitHub.

Speaking about that, I also don't want to get mail from Bevedere stating that I, in fact, have signed a CLA any time I open a PR.

The codecov config lives at https://github.com/python/cpython/blob/master/.github/codecov.yml and the docs for the file are at https://docs.codecov.io/docs/codecov-yaml. Specifically, the comment feature is covered at https://docs.codecov.io/docs/pull-request-comments and the status check is at https://docs.codecov.io/docs/commit-status.

Nothing has changed in the config, though, so it might be a change related to https://docs.codecov.io/docs/codecov-yaml#section-can-i-name-the-file-codecov-yml and thus the file path to the config might need to be added to the upload step in the following files:

1. https://github.com/python/cpython/blob/master/.github/workflows/coverage.yml
2. https://github.com/python/cpython/blob/master/.travis.yml
3. https://github.com/python/cpython/blob/master/.azure-pipelines/posix-steps.yml#L52

(I would submit a PR myself but I'm still digging myself out from under email after vacation.)

Thanks for the pointer Brett, I'll submit a PR.

New changeset 766b7546a564c8e386a3c31eb06fc1b55e8f5a25 by Ammar Askar in branch 'master':
bpo-39704: Explicitly pass the path to codecov config (GH-18680)
https://github.com/python/cpython/commit/766b7546a564c8e386a3c31eb06fc1b55e8f5a25

Can someone with access check that https://codecov.io/gh/python/cpython/settings/yaml has the right config? The latest run after my PR was pushed through still has the status check running :(

Looks like it was just cached, the latest pull request didn't get a codecov comment nor was it ran on the latest commit: https://github.com/python/cpython/pull/18682

Should this be back-ported so backport pull requests/pull requests to other versioned branches don't get affected by this either?

I don't know if backporting will be needed; probably depends on the CI and whether they always pull from master or the branch that was affected. But I just tried backporting regardless and there's conflicts, so it will have to be done manually.

Just a quick update, I think this is a codecov bug as per here: https://community.codecov.io/t/prs-are-commented-even-with-comment-off/941

The yaml configuration doesn't show up here: https://codecov.io/gh/python/cpython/settings/yaml

While we wait for a response from codecov, we can fix this in the interim by overriding the settings at an organization level. An owner of the Python organization can enable this here: the organization level stuff would be here: https://codecov.io/account/gh/python/yaml

placing

```
comment: off
coverage:
  status:
    changes: off
    project: off
    patch: off
```

as the config should hopefully fix this for now.

Ernest updated the organization configuration to:
---
comment: off
coverage:
  status:
    changes: off
    project: off
    patch: off
---

which gives:

{
    "comment": false, 
    "coverage": {
        "status": {
            "changes": false, 
            "patch": false, 
            "project": false
        }
    }
}

Codecov should no longer add comments to pull requests. I close the issue. Reopen it if Codecov adds *new* comments to pull requests.

Thanks Ernest!

FYI I created bpo-40156: "codecov/patch stills runs on 3.5 and 3.6 branches".

Since explicit is better than implicit: yes, we do need backports.  PRs against 3.5 are getting marked red because of automated codecov complaints.