classification
Title: Correct the parsing of a test case docstring.
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benf_wspdigital, brian, louielu, r.david.murray
Priority: normal Keywords:

Created on 2017-04-27 02:44 by benf_wspdigital, last changed 2017-05-16 21:57 by r.david.murray.

Files
File name Uploaded Description Edit
test_foo.py benf_wspdigital, 2017-04-27 02:44 Unit tests demonstrating the problem
issue30181-fix.cfc063a9.tar.gz benf_wspdigital, 2017-05-12 07:12 Implementation fixing this issue, patches up to commit cfc063a9
Pull Requests
URL Status Linked Edit
PR 1505 open python-dev, 2017-05-09 02:05
Messages (19)
msg292387 - (view) Author: Ben Finney (benf_wspdigital) * Date: 2017-04-27 02:44
The docstring of a test case is not correctly parsed for display.

The attached ‘test_foo.py’ module contains two test case functions. Both docstrings conform to PEP 257 <https://www.python.org/dev/peps/pep-0257/>: they have a single-line synopsis and some extra text in a new paragraph.

However, only one of the functions has its docstring synopsis used in the output:

=====
======================================================================
FAIL: test_lower_returns_expected_code (test_foo.Foo_TestCase)
Should return expected code.
----------------------------------------------------------------------
Traceback (most recent call last):
[…]

======================================================================
FAIL: test_reverse_returns_expected_text (test_foo.Foo_TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
[…]

----------------------------------------------------------------------
Ran 2 tests in 0.001s
=====

This violates the docstring parsing as described in PEP 257. The synopsis should be obtained by, first, stripping leading and trailing whitespace from the docstring; then, from that stripped text, taking the first line as the synopsis.

So the expected output for ‘test_foo.Foo_TestCase. test_reverse_returns_expected_text’ should include its docstring synopsis, “Should return expected reverse text.”
msg292396 - (view) Author: Louie Lu (louielu) * Date: 2017-04-27 04:12
This is because unittest.TestCase method `shortDescription()` will only return the first line of docstring, writing at here*:

"""
The default implementation of this method returns the first line of the test method’s docstring, if available, or None.
"""

Not sure if we may change this default behavior for this.



* https://docs.python.org/3/library/unittest.html#unittest.TestCase.shortDescription
msg292406 - (view) Author: Ben Finney (benf_wspdigital) * Date: 2017-04-27 06:10
> This is because unittest.TestCase method `shortDescription()` will only return the first line of docstring

Yes, that is the “docstring synopsis” I referred to. PEP 257 has a section specifically about how tools should parse a docstring: https://www.python.org/dev/peps/pep-0257/#handling-docstring-indentation

In particular, note that “ Blank lines should be removed from the beginning and end of the docstring.”

So, the “first line of the docstring“ for the example ‘test_reverse_returns_expected_text’ is the text “Should return expected reverse text.”.
msg292427 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-04-27 11:49
Considering that in the cpython test suite we avoid using docstrings in unittests because of this "feature" of unittest, I doubt anyone on the core team is going to be motivated to fix this :)  That doesn't mean we won't accept a PR, but if we do we would only put it in 3.7 because it is a visible behavior change, and there are tools that parse unittest output.
msg292649 - (view) Author: Ben Finney (benf_wspdigital) * Date: 2017-05-01 07:23
Howdy R. David,

Okay, I have put together a series of commits to address this bug. The
changes are in my personal fork of the ‘cpython’ repository, so this is
a pull request from ‘wip/issue/issue30181_parse-docstring-using-pydoc’
in that repository.

The following changes since commit 8550803dfba4cf7018ee7415593ceb34c52923e6:

  Backed out changeset f23fa1f7b68f (2017-02-10 14:19:36 +0100)

are available in the git repository at:

  https://benf_wspdigital@bitbucket.org/benf_wspdigital/cpython.git

for you to fetch changes up to c2de731582a9da1147a300a991890ff663891e75:

  Parse the test case docstring using `pydoc.splitdoc`. (2017-05-01 16:55:52 +1000)

----------------------------------------------------------------
(from the branch description for wip/issue/issue30181_parse-docstring-using-pydoc local branch)

Correct the parsing of a test case docstring.

----------------------------------------------------------------
Ben Finney (4):
      Add test cases for single-line test case docstring.
      Add test cases for stripping surrounding space for shortDescription.
      Add test cases for test case docstring with surrounding newlines.
      Parse the test case docstring using `pydoc.splitdoc`.

 Lib/unittest/case.py                       | 34 ++++++++++++------
 Lib/unittest/test/test_case.py             | 42 ++++++++++++++++++++++
 Lib/unittest/test/test_functiontestcase.py | 58 +++++++++++++++++++++++++-----
 3 files changed, 115 insertions(+), 19 deletions(-)
msg292650 - (view) Author: Louie Lu (louielu) * Date: 2017-05-01 07:45
Ben, the process of submitting PR was migrated to GitHub, you will need to use a GitHub account to do it.

You can refer to devguide for how to submit a PR here:

http://cpython-devguide.readthedocs.io/pullrequest.html
msg292651 - (view) Author: Ben Finney (benf_wspdigital) * Date: 2017-05-01 07:54
On 2017-05-01 17:45, Louie Lu wrote:
> Ben, the process of submitting PR was migrated to GitHub

Thank you. I'll re-base my branch onto the master branch found at the GitHub repository.

> you will need to use a GitHub account to do it

I don't maintain repositories at GitHub. Does that mean the contribution must be rejected?
msg292652 - (view) Author: Louie Lu (louielu) * Date: 2017-05-01 08:42
> Thank you. I'll re-base my branch onto the master branch found at the GitHub repository.

You may re-clone a GitHub repository instead of rebase bitbucket repo.


> I don't maintain repositories at GitHub. Does that mean the contribution must be rejected?

No, your contribution will not be rejected by "you don't maintain the repo on GitHub", everyone can send the patch (PR) to python/cpython.

you will need to fork python/cpython from GitHub first, this will create a fork in your GitHub account, maybe benf/cpython if your GitHub account is benf, and then clone this repo (benf/cpython), the rest step please see the instruction in devguide !
msg292654 - (view) Author: Louie Lu (louielu) * Date: 2017-05-01 08:44
Ben, if you have any problem about how to get on with GitHub, I can help you at #python-dev @ freenode, ping louielu on the chatroom.
msg293268 - (view) Author: Brian May (brian) * Date: 2017-05-09 02:25
I made an attempt at a PR. Unfortunately some of the tests appear to fail on some environments, and the messages produced don't exactly clarify why.
msg293271 - (view) Author: Brian May (brian) * Date: 2017-05-09 02:47
Me getting confused at the newline in the comparison output.
msg293522 - (view) Author: Ben Finney (benf_wspdigital) * Date: 2017-05-12 04:33
Apparently there is some consternation about my bugs.python.org identity. This is Ben Finney of WSP Digital, in Melbourne, Australia.

A GitHub account is wholly irrelevant to this account. A GitHub account is wholly irrelevant to accepting the PSF CLA.

This bugs.python.org account `benf_wspdigital` does not mention a GitHub account, and that's the way I want to keep it.
msg293523 - (view) Author: Ben Finney (benf_wspdigital) * Date: 2017-05-12 04:39
> No, your contribution will not be rejected by "you don't maintain the repo on GitHub", everyone can send the patch (PR) to python/cpython.

I am glad to know that. I won't be maintaining a GitHub account because I disagree with the terms of service.

> Ben, if you have any problem about how to get on with GitHub, I can help you […]

Thank you for the offer. My problem with how to get on with GitHub is I don't accept their terms of service :-) How do we proceed?

Git is a *decentralised* VCS, it should not be a barrier that I make the contribution from a different repository host.
msg293525 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-05-12 05:50
Our tooling and workflow is indeed dependent on github.  We should repurpose the current 'remote hg repo' to allow the entry of a non-github git repo, but someone will have to write that code.  In the meantime, if you upload a patch generated from your branch here, someone else will create the github PR so we can get it processed.  If you generate the patch using 'git show -v', the authorship information should get imported into the github PR that gets generated.
msg293527 - (view) Author: Brian May (brian) * Date: 2017-05-12 07:05
I have already created such a pull requests. It is bidirectionally linked to this ticket.

However concerns have been raised that the automatic bot hasn't recognised that Ben Finney has signed the CLA. Hence it appears the pull request might be rejected.

Sure there are currently problems with the pull request, and it fails the tests, however I am not going to spend any additional efforts fixing them unless this situation with the CLA can be resolved first.
msg293528 - (view) Author: Ben Finney (benf_wspdigital) * Date: 2017-05-12 07:12
> if you upload a patch generated from your branch here, someone else will create the github PR so we can get it processed.

The branch has several commits. I have generated the patches, and am now uploading them as a tarball.

It may be easier to simply merge from the Git branch, https://rocketgit.com/user/benf_wspdigital/cpython/source/log/branch/wip,issue,issue30181_parse-docstring-using-pydoc
msg293561 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-05-12 17:23
Ah, yes, that's a bug in our workflow we'll need to fix.  The CLA signed on the bug tracker is the ultimate authority, so we need some way to manually bypass the bot for cases like this.

A committer should be able to bypass the problem and get the patch pushed, once it is approved.
msg293681 - (view) Author: Ben Finney (benf_wspdigital) * Date: 2017-05-15 06:53
> I have already created such a pull requests. It is bidirectionally linked to this ticket.

I hope the PSF can start accepting contributions without that laundering, but until then: thank you for taking that on.
msg293804 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-05-16 21:57
It's not the PSF, it's the dev team, and it is "just" a matter of tooling.  Which means help improving the tooling is welcome :)
History
Date User Action Args
2017-05-16 21:57:49r.david.murraysetmessages: + msg293804
2017-05-15 06:53:04benf_wspdigitalsetmessages: + msg293681
2017-05-12 17:23:04r.david.murraysetmessages: + msg293561
2017-05-12 07:12:23benf_wspdigitalsetfiles: + issue30181-fix.cfc063a9.tar.gz

messages: + msg293528
2017-05-12 07:05:27briansetmessages: + msg293527
2017-05-12 05:50:41r.david.murraysetmessages: + msg293525
2017-05-12 04:39:05benf_wspdigitalsetmessages: + msg293523
2017-05-12 04:33:42benf_wspdigitalsetmessages: + msg293522
2017-05-09 02:47:51briansetmessages: + msg293271
2017-05-09 02:25:11briansetmessages: + msg293268
2017-05-09 02:05:47python-devsetpull_requests: + pull_request1606
2017-05-01 08:44:48louielusetmessages: + msg292654
2017-05-01 08:42:26louielusetmessages: + msg292652
2017-05-01 07:54:20benf_wspdigitalsetmessages: + msg292651
2017-05-01 07:45:27louielusetmessages: + msg292650
2017-05-01 07:23:05benf_wspdigitalsetmessages: + msg292649
title: Incorrect parsing of test case docstring -> Correct the parsing of a test case docstring.
2017-04-27 11:49:57r.david.murraysetversions: - Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6
nosy: + r.david.murray

messages: + msg292427

type: behavior -> enhancement
2017-04-27 06:10:11benf_wspdigitalsetmessages: + msg292406
2017-04-27 04:12:17louielusetnosy: + louielu
messages: + msg292396
2017-04-27 03:36:38briansetnosy: + brian
2017-04-27 02:44:34benf_wspdigitalcreate