classification
Title: Document how to run coverage for repository idlelib files.
Type: enhancement Stage: patch review
Components: IDLE Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: cheryl.sabella, louielu, taleinat, terry.reedy
Priority: normal Keywords: patch

Created on 2017-07-15 01:19 by terry.reedy, last changed 2020-10-14 14:18 by taleinat.

Pull Requests
URL Status Linked Edit
PR 2711 merged terry.reedy, 2017-07-15 02:27
PR 2726 merged terry.reedy, 2017-07-16 04:09
PR 2733 closed louielu, 2017-07-16 13:10
PR 22694 open taleinat, 2020-10-14 14:17
Messages (12)
msg298379 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-15 01:19
I have been using coverage for a few years. Now that I am (again) not the only one writing IDLE tests (others did 3 and 4 years ago), I should document what I do and in particular the policy decision -- test with 'branch=True' -- and the IDLE-specific excludes:
    .*# htest #
    if not _utest:
    if _htest:
msg298384 - (view) Author: Louie Lu (louielu) * Date: 2017-07-15 05:22
After merging this one, I can provide the version of Linux and MacOS.
msg298386 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-15 05:52
That would be good; I might need to know someday;-).
msg298412 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-16 04:07
New changeset 95bebb7264afad88f31f7cdc3b28bacde7c46f93 by terryjreedy in branch 'master':
bpo-30934: Document coverage details for idlelib tests (#2711)
https://github.com/python/cpython/commit/95bebb7264afad88f31f7cdc3b28bacde7c46f93
msg298415 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-16 04:40
New changeset 65de1f3672fd0a1e3ec34b654ef5213e15a03e26 by terryjreedy in branch '3.6':
[3.6] bpo-30934: Document coverage details for idlelib tests (GH-2711) (#2726)
https://github.com/python/cpython/commit/65de1f3672fd0a1e3ec34b654ef5213e15a03e26
msg298416 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-16 04:41
I generalized the first part so only the .bat file needs a *nix version.
msg298436 - (view) Author: Louie Lu (louielu) * Date: 2017-07-16 13:11
Upload the Linux/MacOS version of coverage usage.
msg298446 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-16 19:23
Cheryl, please try Louie's linux instructions and see if they work on Ubuntu.
msg298494 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2017-07-17 12:47
I tried out the instructions.  I guess my first question is how stand alone these instructions are and how much you expect someone to already know about coverage.  What I mean is, I needed to go to the devguide to read more about coverage in order to run the steps here.

For example, the devguide suggests creating a venv before running pip install coverage.  If I do that and go into the venv, then I could do (with python instead of ./python):

python -m coverage run --source Lib/idlelib Lib/idlelib/idle_test/test_config.py
python -m coverage report
python -m coverage html

Those steps seem to work.

Without the venv, ./python -m coverage doesn't work without the pip install (I get the 'No module named coverage' error).  At some point (as per the devguide), I had also downloaded the in-development version, so I am able to run 
./python ../coveragepy run --source Lib/idlelib Lib/idlelib/idle_test/test_config.py

At the time, the devguide suggested the in development version because it was more recent for 3.7 than the released version was, but it seems that the version on pip is the most current now.

Just like everything else, it seems that there are a few ways to go with this, but the venv with pip install seems to be the most foolproof.  I don't know if you'd also want to include instructions for running coverage over all of idle_test instead of just one module.


(without the ./python)
msg298569 - (view) Author: Louie Lu (louielu) * Date: 2017-07-18 02:14
I update how to prepare coverage on Linux and MacOS. I first thought user will prepare as Terry wrote at the top of `5. Test Coverage`. 

Cheryl's method is good, the update guide is borrow from here. First create a virtualenv, then install coverage via pip in venv. If user want to skip the step of venv, they can copy out the coverage package in venv/lib/ to Lib/ inside.
msg378580 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-10-13 21:45
Tal, this is an existing issue about documenting how to run IDLE coverage.  It was left open to add something about the *nix equivalent of my Windows instructions.  I would like to finish this, with updates elsewhere in the text.

I'm dubious about Louie's patch (PR 2733) for multiple reasons: it starts with venv's, which I consider a separate issue; it gives the commands that might go in a script, but does not comprise a script itself; I cannot tell if it uses an IDLE specific .coverage.  Louie ceased IDLE involvement in Sept 2017, so I am closing his patch in favor of whatever you might contribute.

The .coveragerc copied into the README is missing some additions, and I plan to add some annotations to the original.  Do you think it better to have a copy in README.txt, or a separate .coveragerc file in idle_test?  Same question for cover.bat.

"5. Test Coverage" begins with "Install the coverage package into your Python 3.6 site-packages directory."  That should be '3.x' or 'the most recent installed Python possible', so it is most likely to be compatible with current master branch python.  I should also mention that we only need coverage on master, and have other revisions in mind.

Cheryl mentioned above instructions for coverage of all IDLE tests.  I have the command, commented out, in my original cover.bat.  After retesting, I could add it, commented out, to the copy.
msg378619 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2020-10-14 14:18
See a cross-platform script in PR GH-22694.
History
Date User Action Args
2020-10-14 14:18:48taleinatsetmessages: + msg378619
2020-10-14 14:17:13taleinatsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request21665
2020-10-13 21:45:25terry.reedysetnosy: + taleinat, - wohlganger
messages: + msg378580
versions: + Python 3.8, Python 3.9, Python 3.10, - Python 3.6, Python 3.7
2017-07-18 02:14:04louielusetmessages: + msg298569
2017-07-17 12:47:37cheryl.sabellasetmessages: + msg298494
2017-07-16 19:23:05terry.reedysetmessages: + msg298446
2017-07-16 13:11:04louielusetmessages: + msg298436
2017-07-16 13:10:16louielusetpull_requests: + pull_request2794
2017-07-16 04:41:03terry.reedysetmessages: + msg298416
2017-07-16 04:40:02terry.reedysetmessages: + msg298415
2017-07-16 04:09:38terry.reedysetpull_requests: + pull_request2786
2017-07-16 04:07:38terry.reedysetmessages: + msg298412
2017-07-15 05:52:30terry.reedysetmessages: + msg298386
2017-07-15 05:22:38louielusetmessages: + msg298384
2017-07-15 02:27:18terry.reedysetpull_requests: + pull_request2775
2017-07-15 01:19:45terry.reedycreate