classification
Title: Derby #16: Convert 50 sites to Argument Clinic across 9 files
Type: enhancement Stage: needs patch
Components: Argument Clinic, Extension Modules Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: larry, nadeem.vawda, ncoghlan, python-dev, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2014-01-08 00:21 by larry, last changed 2015-05-05 06:04 by serhiy.storchaka.

Files
File name Uploaded Description Edit
issue20184_builtin_conversion.diff ncoghlan, 2014-01-25 09:44 Approach to ensuring all builtins have signatures review
issue20184_builtin_conversion_v2.diff ncoghlan, 2014-01-25 15:12 All non-type callables in bltinmodule.c converted or classified review
issue20184_builtin_conversion_v3.diff ncoghlan, 2014-01-27 07:35 Updated to fix test_pydoc and test_gdb review
issue20184_builtin_conversion_v4.diff ncoghlan, 2014-02-02 06:38 Updated for AC changes review
issue20184_builtin_conversion_v5.diff ncoghlan, 2014-02-02 06:42 Additional cleanups in the test case review
issue20184_builtin_conversion_v6.diff ncoghlan, 2014-03-30 05:00 Updated for AC changes in the lead up to 3.4.0 review
issue20184_builtin_conversion_v7.diff ncoghlan, 2014-08-16 07:57 Updated to apply to trunk (August 2014) review
dbm_clinic.patch serhiy.storchaka, 2015-04-15 11:57 review
dbm_clinic_2.patch serhiy.storchaka, 2015-04-16 15:51 review
json_clinic.patch serhiy.storchaka, 2015-05-05 06:04 review
Messages (18)
msg207642 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-01-08 00:21
This issue is part of the Great Argument Clinic Conversion Derby,
where we're trying to convert as much of Python 3.4 to use
Argument Clinic as we can before Release Candidate 1 on January 19.

This issue asks you to change the following bundle of files:
    Modules/faulthandler.c: 7 sites
    Modules/_pickle.c: 7 sites
    Modules/_lzmamodule.c: 7 sites
    Python/bltinmodule.c: 6 sites
    Modules/termios.c: 6 sites
    Modules/syslogmodule.c: 6 sites
    Modules/_dbmmodule.c: 4 sites
    Modules/_gdbmmodule.c: 2 sites
    Modules/_json.c: 5 sites

Talk to me (larry) if you only want to attack part of a bundle.

For instructions on how to convert a function to work with Argument
Clinic, read the "howto":
    http://docs.python.org/dev/howto/clinic.html
msg209159 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-01-25 07:08
I'm going to specifically tackle the "bltinmodule" side of things using a test driven approach: adding a new test to test_inspect that checks all the builtin names have signatures (I'll explicitly exclude ones that are known not to be handled, like range and slice).
msg209179 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-01-25 09:44
Attached patch shows the new test case I'm using to ensure that all callable builtins have signatures, and once we get it that way, it stays that way.

Preliminary goal is signatures for all the non-type objects, and once I get to that point, I'll propose it for review.
msg209203 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-01-25 15:12
More comprehensive patch uploaded - all the non-type callables implemented in bltinmodule.c have been converted or classified with a reason for not being converted yet (see the new test in test_inspect.py for details, as well as the AC 3.4 and AC 3.5 comments in the module itself).

I also cleaned up the docstrings for the builtins I actually changed in the patch. There were a few that had never been properly updated for the Py3k transition.

There are still a couple of test failures with this version - the doctest tests get confused by the fact ord and chr now have a doctest in their docstrings, and test_gdb is definitely not in a happy place (that has always been temperamental, though).

I also just realised the Unicode character in the new ord and chr docstrings could pose a compatibility problem at the C compiler source encoding level, so we may have to reconsider that (even though I was rather happy to sneak that obscure Monty Python reference in there).
msg209409 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-01-27 07:35
Assigning to myself to make it clear that bltinmodule is the only part of this still under consideration for 3.4.

The test_pydoc and test_gdb failures pointed to real issues with the previous patch:

- the pydoc errors themselves were incidental, indicating that I had added doctests to chr and ord. However, those new doctests used a Unicode character in a C string, which seems like a recipe for portability trouble. I took those doctests out again, and updated the prose docs instead. I like my obscure multilingual Monty Python reference and would like to keep it now I thought of it :)

- the gdb error suggests that gdb is relying on being able to find builtin_id based on its exact signature, including the parameter names. Rather than trying to figure out the full details of that, I've instead partially reverted its conversion to Argument Clinic by disabling the input block and restoring the old parameter names in the function signature. We can change it back to full conversion for 3.5, after AC has the ability to use different names in the Python signature and in the C implementation function.

This does mean we'll want to update the signature by hand once you merge the patch changing the indicators that argument clinic is looking for.
msg209953 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-02-02 06:38
Larry, if this version looks good to you, I'd like to commit it.

- id() is now back to being a properly generated AC function (since AC can now preserve the old C level signature)

- sorted() is partially converted and has a __text_signature__ compatible docstring. However, full conversion will have to wait for the ability to preserve the kwds dict, since that's the API exposed by the list object.

- with the new never-triggered-by-accident AC syntax, the test case now ensures that all the builtins that are expected to *not* expose signature info at this point, don't. As they're converted for 3.5, that will force them to be added to the list of functions that are checked for compatibility.

I'm thinking it's probably worth flagging this new test as a CPython implementation detail test, though.
msg210218 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-02-04 12:22
Larry, I'd like to include the most-builtin-functions signature support for 3.4, but since the AC Derby is over, it's your call as release manager.

If it looks good to you, go ahead and commit it (I won't have time until later in the week), otherwise just drop the priority and punt it to 3.5.
msg210375 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-02-06 11:17
Interpreting the lack of response as "this can wait until 3.5".
msg215164 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-03-30 05:00
Updated the builtins patch for the 3.4.0 Argument Clinic changes. Also moved the issue back to 3.4, since Larry has indicated that AC conversions are in scope for 3.4.x maintenance releases, so long as no public API behaviour changes.
msg224765 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-08-04 20:14
All the Derby patches should only go into trunk at this point.
msg225385 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-08-16 07:57
Tweaked patch to apply cleanly to trunk, but realised it has been a while since I looked at the current state of argument clinic:

* I'm not sure if varargs support has been added yet. If it has, "min", "max", "print", "__build_class__" can be converted. If it hasn't, then the comments referring to 3.4 still need to be updated.

* the comments referring to needing groups support in AC aren't quite right, it's optional group support in inspect that's missing. So the comments relating to "range", "slice", "dir", "getattr", "next", "iter", "vars" should be tweaked accordingly

* tweaking the behaviour of round to make it AC friendly should probably go into its own issue.

Also, Larry, there's a note in the the 3.4 "What's New" about additional signature info landing in Python 3.4 maintenance releases. That note should probably be adjusted at this point, since any new signature data will only be available in 3.5.
msg225423 - (view) Author: Roundup Robot (python-dev) Date: 2014-08-17 04:11
New changeset 6219aa966a5f by Nick Coghlan in branch 'default':
Issue #20184: Add signature introspection for 30 of the builtins
http://hg.python.org/cpython/rev/6219aa966a5f
msg225424 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-08-17 04:14
30 of the builtin functions now support signature introspection thanks to Argument Clinic :)
msg225425 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-08-17 04:14
(Well, they'll support it in 3.5...)
msg241094 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-15 11:57
Proposed patch converts _dbm and _gdbm modules to Argument Clinic.
msg241226 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-16 15:51
The patch updated to the tip.
msg241347 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-17 18:06
New changeset 49910ff21ba5 by Serhiy Storchaka in branch 'default':
Issue #20184: Converted _dbm and _gdbm modules to Argument Clinic.
https://hg.python.org/cpython/rev/49910ff21ba5
msg242588 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-05-05 06:04
Here is a patch that converts 3 functions in the _json module to Argument Clinic. All other functions doesn't fit with Argument Clinic.
History
Date User Action Args
2015-05-05 06:04:04serhiy.storchakasetfiles: + json_clinic.patch

messages: + msg242588
2015-04-17 18:06:47python-devsetmessages: + msg241347
2015-04-16 15:51:57serhiy.storchakasetfiles: + dbm_clinic_2.patch

messages: + msg241226
2015-04-15 11:57:12serhiy.storchakasetfiles: + dbm_clinic.patch
nosy: + serhiy.storchaka
messages: + msg241094

2015-02-25 15:28:52serhiy.storchakasetcomponents: + Argument Clinic
2014-08-17 04:14:47ncoghlansetmessages: + msg225425
2014-08-17 04:14:09ncoghlansetmessages: + msg225424
2014-08-17 04:11:47python-devsetnosy: + python-dev
messages: + msg225423
2014-08-16 07:58:05ncoghlansetfiles: + issue20184_builtin_conversion_v7.diff

messages: + msg225385
2014-08-04 20:14:49larrysetmessages: + msg224765
versions: + Python 3.5, - Python 3.4
2014-03-30 05:00:41ncoghlansetfiles: + issue20184_builtin_conversion_v6.diff
versions: + Python 3.4, - Python 3.5
nosy: + yselivanov

messages: + msg215164
2014-02-06 11:17:12ncoghlansetpriority: release blocker -> normal
assignee: larry ->
messages: + msg210375

versions: + Python 3.5, - Python 3.4
2014-02-04 12:22:28ncoghlansetpriority: normal -> release blocker
assignee: ncoghlan -> larry
messages: + msg210218
2014-02-02 06:42:38ncoghlansetfiles: + issue20184_builtin_conversion_v5.diff
2014-02-02 06:38:42ncoghlansetfiles: + issue20184_builtin_conversion_v4.diff

messages: + msg209953
2014-01-27 07:35:37ncoghlansetfiles: + issue20184_builtin_conversion_v3.diff
assignee: ncoghlan
messages: + msg209409
2014-01-25 15:12:40ncoghlansetfiles: + issue20184_builtin_conversion_v2.diff

messages: + msg209203
2014-01-25 09:44:24ncoghlansetfiles: + issue20184_builtin_conversion.diff
keywords: + patch
messages: + msg209179
2014-01-25 07:08:58ncoghlansetnosy: + ncoghlan
messages: + msg209159
2014-01-08 11:32:38nadeem.vawdasetnosy: + nadeem.vawda
2014-01-08 01:36:37r.david.murraylinkissue20187 dependencies
2014-01-08 00:21:44larrycreate