classification
Title: Code coverage report not entirely accurate
Type: enhancement Stage: resolved
Components: Tests Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Lewis Ball, brett.cannon, terry.reedy
Priority: normal Keywords: patch

Created on 2020-05-02 01:11 by Lewis Ball, last changed 2020-05-28 18:52 by brett.cannon. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19851 merged Lewis Ball, 2020-05-02 01:15
Messages (5)
msg367904 - (view) Author: Lewis Ball (Lewis Ball) * Date: 2020-05-02 01:11
The current reported code coverage stats on codecov.io miss some global statements of modules imported before coveragepy begins. This is documented in the dev guide, and the suggested fix is here: https://devguide.python.org/coverage/#coverage-results-for-modules-imported-early-on but that recommended fix doesn't seem to be included in the CI.

Using the recommended 'hack', the coverage report shows an extra 4k lines are coverage in Lib, increasing coverage percentage by around 3%. 

Having a more accurate view of the coverage makes it much easier to see which parts of the code need help with tests, without having to go into every individual report to see if global statements are misreported.

I'll raise a PR for this shortly
msg368204 - (view) Author: Lewis Ball (Lewis Ball) * Date: 2020-05-05 20:19
I've updated the travis.yml now to fix this issue and global statements are now showing as covered in the coverage report.  This means an extra 4k lines show as covered which weren't previously showing. See the report for the PR here (https://codecov.io/gh/python/cpython/tree/64d521b5d34c25b83d0472608d1eab3a6334bf59/Lib).
msg368460 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-05-08 20:21
Use git to find out who previously edited the file and request reviews from at least a couple.  The 2 recommendations on the PR may be based on that, but good to check.
msg368802 - (view) Author: Lewis Ball (Lewis Ball) * Date: 2020-05-13 21:50
I don't have triage permissions so I don't think I can request a review. Although @aeros has requested one.
msg370260 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-05-28 18:51
New changeset d9c1f1991969e99791de75b2bc935e6445bc5dcd by lrjball in branch 'master':
bpo-40474: Updated coverage.yml to better report coverage stats (#19851)
https://github.com/python/cpython/commit/d9c1f1991969e99791de75b2bc935e6445bc5dcd
History
Date User Action Args
2020-05-28 18:52:33brett.cannonsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-05-28 18:51:51brett.cannonsetnosy: + brett.cannon
messages: + msg370260
2020-05-13 21:50:29Lewis Ballsetmessages: + msg368802
2020-05-08 20:21:49terry.reedysetnosy: + terry.reedy
messages: + msg368460
2020-05-05 20:19:32Lewis Ballsetmessages: + msg368204
2020-05-02 01:15:24Lewis Ballsetkeywords: + patch
stage: patch review
pull_requests: + pull_request19168
2020-05-02 01:11:22Lewis Ballcreate