Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derby #16: Convert 50 sites to Argument Clinic across 9 files #64383

Closed
larryhastings opened this issue Jan 8, 2014 · 25 comments
Closed

Derby #16: Convert 50 sites to Argument Clinic across 9 files #64383

larryhastings opened this issue Jan 8, 2014 · 25 comments
Labels
extension-modules C modules in the Modules dir topic-argument-clinic type-feature A feature request or enhancement

Comments

@larryhastings
Copy link
Contributor

BPO 20184
Nosy @ncoghlan, @larryhastings, @serhiy-storchaka, @1st1, @shihai1991
PRs
  • gh-64383: Migrate builtins.vars and builtins.dir to Argument Clinic #18512
  • bpo-20184: Convert termios to Argument Clinic. #22693
  • Files
  • issue20184_builtin_conversion.diff: Approach to ensuring all builtins have signatures
  • issue20184_builtin_conversion_v2.diff: All non-type callables in bltinmodule.c converted or classified
  • issue20184_builtin_conversion_v3.diff: Updated to fix test_pydoc and test_gdb
  • issue20184_builtin_conversion_v4.diff: Updated for AC changes
  • issue20184_builtin_conversion_v5.diff: Additional cleanups in the test case
  • issue20184_builtin_conversion_v6.diff: Updated for AC changes in the lead up to 3.4.0
  • issue20184_builtin_conversion_v7.diff: Updated to apply to trunk (August 2014)
  • dbm_clinic.patch
  • dbm_clinic_2.patch
  • json_clinic.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2014-01-08.00:21:44.691>
    labels = ['extension-modules', 'type-feature', 'expert-argument-clinic']
    title = 'Derby #16: Convert 50 sites to Argument Clinic across 9 files'
    updated_at = <Date 2020-10-18.14:54:14.974>
    user = 'https://github.com/larryhastings'

    bugs.python.org fields:

    activity = <Date 2020-10-18.14:54:14.974>
    actor = 'serhiy.storchaka'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Extension Modules', 'Argument Clinic']
    creation = <Date 2014-01-08.00:21:44.691>
    creator = 'larry'
    dependencies = []
    files = ['33696', '33701', '33740', '33860', '33861', '34664', '36385', '39032', '39070', '39292']
    hgrepos = []
    issue_num = 20184
    keywords = ['patch']
    message_count = 20.0
    messages = ['207642', '209159', '209179', '209203', '209409', '209953', '210218', '210375', '215164', '224765', '225385', '225423', '225424', '225425', '241094', '241226', '241347', '242588', '362008', '378872']
    nosy_count = 7.0
    nosy_names = ['ncoghlan', 'larry', 'nadeem.vawda', 'python-dev', 'serhiy.storchaka', 'yselivanov', 'shihai1991']
    pr_nums = ['18512', '22693']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue20184'
    versions = ['Python 3.5']

    @larryhastings
    Copy link
    Contributor Author

    This issue is part of the Great Argument Clinic Conversion Derby,
    where we're trying to convert as much of Python 3.4 to use
    Argument Clinic as we can before Release Candidate 1 on January 19.

    This issue asks you to change the following bundle of files:
    Modules/faulthandler.c: 7 sites
    Modules/_pickle.c: 7 sites
    Modules/_lzmamodule.c: 7 sites
    Python/bltinmodule.c: 6 sites
    Modules/termios.c: 6 sites
    Modules/syslogmodule.c: 6 sites
    Modules/_dbmmodule.c: 4 sites
    Modules/_gdbmmodule.c: 2 sites
    Modules/_json.c: 5 sites

    Talk to me (larry) if you only want to attack part of a bundle.

    For instructions on how to convert a function to work with Argument
    Clinic, read the "howto":
    http://docs.python.org/dev/howto/clinic.html

    @larryhastings larryhastings added extension-modules C modules in the Modules dir type-feature A feature request or enhancement labels Jan 8, 2014
    @ncoghlan
    Copy link
    Contributor

    I'm going to specifically tackle the "bltinmodule" side of things using a test driven approach: adding a new test to test_inspect that checks all the builtin names have signatures (I'll explicitly exclude ones that are known not to be handled, like range and slice).

    @ncoghlan
    Copy link
    Contributor

    Attached patch shows the new test case I'm using to ensure that all callable builtins have signatures, and once we get it that way, it stays that way.

    Preliminary goal is signatures for all the non-type objects, and once I get to that point, I'll propose it for review.

    @ncoghlan
    Copy link
    Contributor

    More comprehensive patch uploaded - all the non-type callables implemented in bltinmodule.c have been converted or classified with a reason for not being converted yet (see the new test in test_inspect.py for details, as well as the AC 3.4 and AC 3.5 comments in the module itself).

    I also cleaned up the docstrings for the builtins I actually changed in the patch. There were a few that had never been properly updated for the Py3k transition.

    There are still a couple of test failures with this version - the doctest tests get confused by the fact ord and chr now have a doctest in their docstrings, and test_gdb is definitely not in a happy place (that has always been temperamental, though).

    I also just realised the Unicode character in the new ord and chr docstrings could pose a compatibility problem at the C compiler source encoding level, so we may have to reconsider that (even though I was rather happy to sneak that obscure Monty Python reference in there).

    @ncoghlan
    Copy link
    Contributor

    Assigning to myself to make it clear that bltinmodule is the only part of this still under consideration for 3.4.

    The test_pydoc and test_gdb failures pointed to real issues with the previous patch:

    • the pydoc errors themselves were incidental, indicating that I had added doctests to chr and ord. However, those new doctests used a Unicode character in a C string, which seems like a recipe for portability trouble. I took those doctests out again, and updated the prose docs instead. I like my obscure multilingual Monty Python reference and would like to keep it now I thought of it :)

    • the gdb error suggests that gdb is relying on being able to find builtin_id based on its exact signature, including the parameter names. Rather than trying to figure out the full details of that, I've instead partially reverted its conversion to Argument Clinic by disabling the input block and restoring the old parameter names in the function signature. We can change it back to full conversion for 3.5, after AC has the ability to use different names in the Python signature and in the C implementation function.

    This does mean we'll want to update the signature by hand once you merge the patch changing the indicators that argument clinic is looking for.

    @ncoghlan ncoghlan self-assigned this Jan 27, 2014
    @ncoghlan
    Copy link
    Contributor

    ncoghlan commented Feb 2, 2014

    Larry, if this version looks good to you, I'd like to commit it.

    • id() is now back to being a properly generated AC function (since AC can now preserve the old C level signature)

    • sorted() is partially converted and has a __text_signature__ compatible docstring. However, full conversion will have to wait for the ability to preserve the kwds dict, since that's the API exposed by the list object.

    • with the new never-triggered-by-accident AC syntax, the test case now ensures that all the builtins that are expected to *not* expose signature info at this point, don't. As they're converted for 3.5, that will force them to be added to the list of functions that are checked for compatibility.

    I'm thinking it's probably worth flagging this new test as a CPython implementation detail test, though.

    @ncoghlan
    Copy link
    Contributor

    ncoghlan commented Feb 4, 2014

    Larry, I'd like to include the most-builtin-functions signature support for 3.4, but since the AC Derby is over, it's your call as release manager.

    If it looks good to you, go ahead and commit it (I won't have time until later in the week), otherwise just drop the priority and punt it to 3.5.

    @ncoghlan
    Copy link
    Contributor

    ncoghlan commented Feb 6, 2014

    Interpreting the lack of response as "this can wait until 3.5".

    @ncoghlan
    Copy link
    Contributor

    Updated the builtins patch for the 3.4.0 Argument Clinic changes. Also moved the issue back to 3.4, since Larry has indicated that AC conversions are in scope for 3.4.x maintenance releases, so long as no public API behaviour changes.

    @larryhastings
    Copy link
    Contributor Author

    All the Derby patches should only go into trunk at this point.

    @ncoghlan
    Copy link
    Contributor

    Tweaked patch to apply cleanly to trunk, but realised it has been a while since I looked at the current state of argument clinic:

    • I'm not sure if varargs support has been added yet. If it has, "min", "max", "print", "__build_class__" can be converted. If it hasn't, then the comments referring to 3.4 still need to be updated.

    • the comments referring to needing groups support in AC aren't quite right, it's optional group support in inspect that's missing. So the comments relating to "range", "slice", "dir", "getattr", "next", "iter", "vars" should be tweaked accordingly

    • tweaking the behaviour of round to make it AC friendly should probably go into its own issue.

    Also, Larry, there's a note in the the 3.4 "What's New" about additional signature info landing in Python 3.4 maintenance releases. That note should probably be adjusted at this point, since any new signature data will only be available in 3.5.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Aug 17, 2014

    New changeset 6219aa966a5f by Nick Coghlan in branch 'default':
    Issue bpo-20184: Add signature introspection for 30 of the builtins
    http://hg.python.org/cpython/rev/6219aa966a5f

    @ncoghlan
    Copy link
    Contributor

    30 of the builtin functions now support signature introspection thanks to Argument Clinic :)

    @ncoghlan
    Copy link
    Contributor

    (Well, they'll support it in 3.5...)

    @serhiy-storchaka
    Copy link
    Member

    Proposed patch converts _dbm and _gdbm modules to Argument Clinic.

    @serhiy-storchaka
    Copy link
    Member

    The patch updated to the tip.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Apr 17, 2015

    New changeset 49910ff21ba5 by Serhiy Storchaka in branch 'default':
    Issue bpo-20184: Converted _dbm and _gdbm modules to Argument Clinic.
    https://hg.python.org/cpython/rev/49910ff21ba5

    @serhiy-storchaka
    Copy link
    Member

    Here is a patch that converts 3 functions in the _json module to Argument Clinic. All other functions doesn't fit with Argument Clinic.

    @shihai1991
    Copy link
    Member

    Looks like builtins.vars and builtins.dir should be migrated to AC too, so I create PR18512.

    @serhiy-storchaka
    Copy link
    Member

    New changeset 1bcaa81 by Serhiy Storchaka in branch 'master':
    bpo-20184: Convert termios to Argument Clinic. (GH-22693)
    1bcaa81

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @arhadthedev
    Copy link
    Member

    Is this metaissue still relevant after nine years? Currently, conversions into AC are handled in smaller, more specific issues so it could be closed.

    @carljm
    Copy link
    Member

    carljm commented Feb 20, 2023

    It looks to me like all of these derby issues would be better closed. Even if some functions do remain unconverted, it's much more likely at this point that they will be converted because someone notices the unconverted function in the codebase, not because of a derby issue with non-descriptive (and almost certainly now obsolete) name that requires digging through all of the listed modules to see if there's any work remaining to be done on it.

    As the derby issues say, it was an effort to convert as much as possible before Python 3.4 release. Python 3.4 is long ago released, so the derby is over.

    @carljm carljm closed this as completed Feb 20, 2023
    @carljm
    Copy link
    Member

    carljm commented Feb 20, 2023

    Will leave this open for a bit to see if a core dev agrees before closing all of them.

    @carljm carljm reopened this Feb 20, 2023
    @iritkatriel
    Copy link
    Member

    Will leave this open for a bit to see if a core dev agrees before closing all of them.

    It's been open for long enough now.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    extension-modules C modules in the Modules dir topic-argument-clinic type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    7 participants