classification
Title: Argument Clinic test is broken
Type: behavior Stage: resolved
Components: Argument Clinic, Tests Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: jarondl, larry, serhiy.storchaka, vstinner, zach.ware
Priority: critical Keywords: patch

Created on 2016-05-01 13:20 by serhiy.storchaka, last changed 2018-09-04 05:07 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8821 closed pablogsal, 2018-08-19 16:52
PR 8879 merged vstinner, 2018-08-23 20:59
PR 9048 merged miss-islington, 2018-09-03 21:17
Messages (11)
msg264595 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-01 13:20
$ ./python Tools/clinic/clinic_test.py
Error:
Destination does not exist: 'file'

It works in 3.4.
msg264615 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-05-02 03:12
The clinic tests should probably be integrated into the regrtest suite, since clinic is integrated into the build process.
msg264616 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-05-02 03:15
I will admit that even during development I rarely used Clinic's test suite.  My test suite is: make a change, then run Clinic over all of CPython and do a diff.  If nothing changed then Clinic is still working.
msg286946 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-04 10:02
Any chance to fix Argument Clinic test? Without this we can't add new features, fix bugs and add tests for searching new bugs. This is critical for further converting to Argument Clinic.
msg299720 - (view) Author: Yaron de Leeuw (jarondl) * Date: 2017-08-03 19:05
I have done some diving and git bisecting into this. There are at least two issues:

1. `Clinic.__init__` no longer works without passing in a filename. Because soon after `if filename` in line 1726 [1] it calls `d('file')` which only works if there is a `file` buffer.
This makes the odd line 38 [2] in clinic_test.py to fail miserably with sys.exit. Deleting this line results in a proper test failure (22 test fail out of 54), which is already much better than the sys exit message.

2. `Clinic` has shifted from using `field_destinations` to `destination_buffers` in issue 23500. The test code did not change accordingly.


This means that we need to adapt the test code to the use `destination_buffers`, and add it to the regular test suite. The whole situation could only have happened because the test were not run regularly.

I would like to tackle this, but I am not entirely sure yet what should the differences between FakeClinic and Clinic be. So I am studying it. Hints are welcome.




[1] https://github.com/python/cpython/blob/6969eaf4682beb01bc95eeb14f5ce6c01312e297/Tools/clinic/clinic.py#L1726

[2] https://github.com/python/cpython/blob/6969eaf4682beb01bc95eeb14f5ce6c01312e297/Tools/clinic/clinic_test.py#L38
msg324376 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-30 11:13
Pablo, Serhiy: there are now 2 PR (one written by Pablo, one by me). Would you mind to have a look and tell me what is your preferred PR? My PR is based on Pablo's PR, I just changed how the tests are run to make sure that they are run on *all* CIs, not just Travis CI.
msg324535 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-03 21:17
New changeset 65fc98e7b1f62c2e621f04780a3a77c3498cc195 by Victor Stinner in branch 'master':
bpo-26901: Fix the Argument Clinic test suite (GH-8879)
https://github.com/python/cpython/commit/65fc98e7b1f62c2e621f04780a3a77c3498cc195
msg324538 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-03 21:20
> Any chance to fix Argument Clinic test? Without this we can't add new features, fix bugs and add tests for searching new bugs. This is critical for further converting to Argument Clinic.

Done in master branch. But test_clinic.py is really basic, it should really be extended to add more tests!

I also proposed to backport test_clinic.py to 3.7: PR 9048.

Thanks Pablo for the initial fix and for your review of my PR!
msg324544 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-03 22:24
New changeset 1e921236d7ecc77299d34380d6a2159e9db05a1a by Victor Stinner (Miss Islington (bot)) in branch '3.7':
bpo-26901: Fix the Argument Clinic test suite (GH-8879) (GH-9048)
https://github.com/python/cpython/commit/1e921236d7ecc77299d34380d6a2159e9db05a1a
msg324546 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-03 22:25
We decided to only fix 3.7 and master branches, not 3.6 and older: https://github.com/python/cpython/pull/9048

The initial issue has been fixed, so I close the issue.

A new issue might be opened if someone wants to extend the existing test suite (test_clinic).
msg324553 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-04 05:07
Thank you Pablo and Victor!

I do not mind backporting to 3.6. Argument Clinic still can take fixes, and they can add new tests.
History
Date User Action Args
2018-09-04 05:07:04serhiy.storchakasetmessages: + msg324553
2018-09-03 22:25:34vstinnersetstatus: open -> closed
versions: + Python 3.7, Python 3.8, - Python 3.5, Python 3.6
messages: + msg324546

resolution: fixed
stage: patch review -> resolved
2018-09-03 22:24:20vstinnersetmessages: + msg324544
2018-09-03 21:20:47vstinnersetmessages: + msg324538
2018-09-03 21:17:48miss-islingtonsetpull_requests: + pull_request8510
2018-09-03 21:17:24vstinnersetmessages: + msg324535
2018-08-30 11:13:12vstinnersetmessages: + msg324376
2018-08-23 20:59:55vstinnersetpull_requests: + pull_request8356
2018-08-19 16:52:20pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8300
2017-08-03 19:05:52jarondlsetnosy: + jarondl
messages: + msg299720
2017-02-06 13:16:50vstinnersetnosy: + vstinner
2017-02-04 10:02:31serhiy.storchakasetpriority: high -> critical

messages: + msg286946
2016-05-02 03:15:13larrysetmessages: + msg264616
2016-05-02 03:12:06zach.waresetnosy: + zach.ware
messages: + msg264615
2016-05-01 13:20:07serhiy.storchakacreate