Title: Document how to run coverage for repository idlelib files.
Type: enhancement Stage: needs patch
Components: IDLE Versions: Python 3.7, Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: cheryl.sabella, louielu, terry.reedy, wohlganger
Priority: normal Keywords:

Created on 2017-07-15 01:19 by terry.reedy, last changed 2017-07-18 02:14 by louielu.

Pull Requests
URL Status Linked Edit
PR 2711 merged terry.reedy, 2017-07-15 02:27
PR 2726 merged terry.reedy, 2017-07-16 04:09
PR 2733 open louielu, 2017-07-16 13:10
Messages (10)
msg298379 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-15 01:19
I have been using coverage for a few years. Now that I am (again) not the only one writing IDLE tests (others did 3 and 4 years ago), I should document what I do and in particular the policy decision -- test with 'branch=True' -- and the IDLE-specific excludes:
    .*# htest #
    if not _utest:
    if _htest:
msg298384 - (view) Author: Louie Lu (louielu) * Date: 2017-07-15 05:22
After merging this one, I can provide the version of Linux and MacOS.
msg298386 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-15 05:52
That would be good; I might need to know someday;-).
msg298412 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-16 04:07
New changeset 95bebb7264afad88f31f7cdc3b28bacde7c46f93 by terryjreedy in branch 'master':
bpo-30934: Document coverage details for idlelib tests (#2711)
msg298415 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-16 04:40
New changeset 65de1f3672fd0a1e3ec34b654ef5213e15a03e26 by terryjreedy in branch '3.6':
[3.6] bpo-30934: Document coverage details for idlelib tests (GH-2711) (#2726)
msg298416 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-16 04:41
I generalized the first part so only the .bat file needs a *nix version.
msg298436 - (view) Author: Louie Lu (louielu) * Date: 2017-07-16 13:11
Upload the Linux/MacOS version of coverage usage.
msg298446 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-16 19:23
Cheryl, please try Louie's linux instructions and see if they work on Ubuntu.
msg298494 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2017-07-17 12:47
I tried out the instructions.  I guess my first question is how stand alone these instructions are and how much you expect someone to already know about coverage.  What I mean is, I needed to go to the devguide to read more about coverage in order to run the steps here.

For example, the devguide suggests creating a venv before running pip install coverage.  If I do that and go into the venv, then I could do (with python instead of ./python):

python -m coverage run --source Lib/idlelib Lib/idlelib/idle_test/
python -m coverage report
python -m coverage html

Those steps seem to work.

Without the venv, ./python -m coverage doesn't work without the pip install (I get the 'No module named coverage' error).  At some point (as per the devguide), I had also downloaded the in-development version, so I am able to run 
./python ../coveragepy run --source Lib/idlelib Lib/idlelib/idle_test/

At the time, the devguide suggested the in development version because it was more recent for 3.7 than the released version was, but it seems that the version on pip is the most current now.

Just like everything else, it seems that there are a few ways to go with this, but the venv with pip install seems to be the most foolproof.  I don't know if you'd also want to include instructions for running coverage over all of idle_test instead of just one module.

(without the ./python)
msg298569 - (view) Author: Louie Lu (louielu) * Date: 2017-07-18 02:14
I update how to prepare coverage on Linux and MacOS. I first thought user will prepare as Terry wrote at the top of `5. Test Coverage`. 

Cheryl's method is good, the update guide is borrow from here. First create a virtualenv, then install coverage via pip in venv. If user want to skip the step of venv, they can copy out the coverage package in venv/lib/ to Lib/ inside.
Date User Action Args
2017-07-18 02:14:04louielusetmessages: + msg298569
2017-07-17 12:47:37cheryl.sabellasetmessages: + msg298494
2017-07-16 19:23:05terry.reedysetmessages: + msg298446
2017-07-16 13:11:04louielusetmessages: + msg298436
2017-07-16 13:10:16louielusetpull_requests: + pull_request2794
2017-07-16 04:41:03terry.reedysetmessages: + msg298416
2017-07-16 04:40:02terry.reedysetmessages: + msg298415
2017-07-16 04:09:38terry.reedysetpull_requests: + pull_request2786
2017-07-16 04:07:38terry.reedysetmessages: + msg298412
2017-07-15 05:52:30terry.reedysetmessages: + msg298386
2017-07-15 05:22:38louielusetmessages: + msg298384
2017-07-15 02:27:18terry.reedysetpull_requests: + pull_request2775
2017-07-15 01:19:45terry.reedycreate