Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Argument Clinic should understand *args and **kwargs parameters #64490

Open
larryhastings opened this issue Jan 18, 2014 · 12 comments
Open

Argument Clinic should understand *args and **kwargs parameters #64490

larryhastings opened this issue Jan 18, 2014 · 12 comments
Labels
topic-argument-clinic type-feature A feature request or enhancement

Comments

@larryhastings
Copy link
Contributor

larryhastings commented Jan 18, 2014

BPO 20291
Nosy @rhettinger, @ncoghlan, @vstinner, @larryhastings, @methane, @meadori, @zware, @serhiy-storchaka, @GPHemsley, @isidentical, @Fidget-Spinner, @colorfulappl
PRs
  • gh-64490: varargs support for argument clinic and refactor print with AC #18609
  • bpo-20291: Fix MSVC warnings in getargs.c #27211
  • gh-90350: Optimize builtin functions min() and max() #30286
  • gh-64490: Fix bugs in argument clinic varargs processing #32092
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = None
    created_at = <Date 2014-01-18.01:55:25.863>
    labels = ['type-feature', '3.7', 'expert-argument-clinic']
    title = 'Argument Clinic should understand *args and **kwargs parameters'
    updated_at = <Date 2022-03-24.08:27:39.531>
    user = 'https://github.com/larryhastings'

    bugs.python.org fields:

    activity = <Date 2022-03-24.08:27:39.531>
    actor = 'colorfulappl'
    assignee = 'serhiy.storchaka'
    closed = False
    closed_date = None
    closer = None
    components = ['Demos and Tools', 'Argument Clinic']
    creation = <Date 2014-01-18.01:55:25.863>
    creator = 'larry'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 20291
    keywords = ['patch']
    message_count = 10.0
    messages = ['208380', '225514', '225531', '264959', '285642', '285647', '285654', '285660', '285779', '397721']
    nosy_count = 12.0
    nosy_names = ['rhettinger', 'ncoghlan', 'vstinner', 'larry', 'methane', 'meador.inge', 'zach.ware', 'serhiy.storchaka', 'gphemsley', 'BTaskaya', 'kj', 'colorfulappl']
    pr_nums = ['18609', '27211', '30286', '32092']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue20291'
    versions = ['Python 3.7']

    @larryhastings
    Copy link
    Contributor Author

    Argument Clinic currently prevents the "impl" function from ever seeing the "args" tuple or the "kwargs" dict. There should be a way to ask it to pass those values in to the "impl" function.

    @larryhastings larryhastings added the type-bug An unexpected behavior, bug, or error label Jan 18, 2014
    @larryhastings larryhastings self-assigned this Jan 18, 2014
    @zware zware added type-feature A feature request or enhancement and removed type-bug An unexpected behavior, bug, or error labels Aug 13, 2014
    @larryhastings
    Copy link
    Contributor Author

    So, let's think about this for a minute. What's the API that we *want* here?

    If your function has the signature
    (a, b, c=20, *args)
    and you call it with
    (1, 2, 3, 4, 5)
    should "args" be (4, 5), or (1, 2, 3, 4, 5)?

    I assert that the impl function should get the same "args" (and "kwargs") that a Python function would--that is, post-argument-processing. In the above example "args" should get (4, 5).

    This might be somewhat painful to do in round 1, where we're still leveraging off PyArg_ParseTuple*. But in the future it'll be cheaper to do it this way. In any case, it's the right API, so that's what we should do.

    (Adding Nick just to see if he agrees--he had a use case for *args in the builtin module.)

    @ncoghlan
    Copy link
    Contributor

    Yes, I agree we should follow the Python level semantics, and only capture the excess positional arguments. For the record, the four builtins I flagged as needing this in order to add introspection information:

    __build_class__

    • 2 positional only args, arbitrary additional args
    • checks size with PyTuple_GET_SIZE
    • uses PyTuple_GET_ITEM x2 + PyTuple_GetSlice to unpack

    print

    • only arbitrary position args, iterates and extracts using PyTuple_GetItem
    • uses PyArg_ParseTupleAndKeywords with an empty tuple to extract the keyword-only args

    min
    max

    • use a shared helper function min_max
    • uses the args tuple directly if size > 1
    • otherwise uses PyArg_UnpackTuple to extract the supplied iterable
    • uses PyArg_ParseTupleAndKeywords with an empty tuple to extract the keyword-only args
    • this "one arg means iterable" style API might need to be a special case...

    @serhiy-storchaka
    Copy link
    Member

    I think at first step we can support var-positional parameter only when there are no other positional parameters, and var-keyword parameter only when there are no other keyword parameters. So print, max and dict.update will be supported, but __build_class__, map and functools.partial are not.

    @serhiy-storchaka
    Copy link
    Member

    I'll try to implement the support of var-positional parameters.

    @rhettinger
    Copy link
    Contributor

    In case it is helpful, here's my list of examples where the AC and existing signature objects are insufficiently expressive:

    type(obj)
    type(name, bases, mapping)
        two different signatures depending on type   
    
    range(stop)
    range(start, stop)
    range(start, stop, step)
    
    dict.pop(key[, default])
       default of None has different meaning than missing default
       which raises KeyError when the key is missing
    
    itertools.permutations(iterable[, r])
       where the absence of *r* implies r=len(iterable)

    bisect.bisect_right(a, x[, lo[, hi]]) -> index
    where the absence of *hi* implies hi=len(a)

    min(iterable, *[, default=obj, key=func]) -> value
    min(arg1, arg2, *args, *[, key=func]) -> value
    has two signatures depending on the number of
    positional arguments and a keyword argument
    only used in the first signature. It's implementation
    is also shared with max().

    dict() -> new empty dictionary
    dict(mapping) -> new dictionary initialized from a mapping object's
    (key, value) pairs
    dict(iterable) -> new dictionary initialized as if via:
    d = {}
    for k, v in iterable:
    d[k] = v
    dict(**kwargs) -> new dictionary initialized with the name=value pairs
    in the keyword argument list. For example: dict(one=1, two=2)

    def sumseq(seq, a=0, b=None):
        # Pure python code with nullable int
        if b is None:
            b = len(seq)
        return sum(seq[a:b])

    @vstinner
    Copy link
    Member

    FYI I started to work on a different Argument Clinic enhancement, issue bpo-29299: "Argument Clinic: Fix signature of optional positional-only arguments".

    @serhiy-storchaka
    Copy link
    Member

    Thank you for your examples Raymond, but they don't directly related to this issue, implementing support of var-positional and var-keyword parameters. I believe that it is possible to solve it, and the solution is complex, but is not extremal hard. I'm working on the first part of this.

    Your examples show other, more hard issue. It looks to me that the only solution of that issue is to add support of multiple signatures for functions. But this can break the API of the inspect module.

    @vstinner
    Copy link
    Member

    Once this feature will be implemented, print() should be modified to use Argument Clinic: see the issue bpo-29296.

    @isidentical
    Copy link
    Sponsor Member

    New changeset f88e138 by Ken Jin in branch 'main':
    bpo-20291: Fix MSVC warnings in getargs.c (GH-27211)
    f88e138

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    colorfulappl added a commit to colorfulappl/cpython that referenced this issue Dec 19, 2022
    …o tuple in argument clinic (python#99233)
    
    (cherry picked from commit 69f6cc7)
    kumaraditya303 pushed a commit that referenced this issue Dec 19, 2022
    colorfulappl added a commit to colorfulappl/cpython that referenced this issue Dec 20, 2022
    @gpshead
    Copy link
    Member

    gpshead commented Feb 7, 2023

    #18609 appears to have added *args support, but it wasn't documented.

    • Doc/howto/clinic.rst could use an update.

    @gpshead
    Copy link
    Member

    gpshead commented Feb 7, 2023

    (unassigning since it isn't actively being worked on) @isidentical as FYI for the missing documentation.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    topic-argument-clinic type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    9 participants