Title: Argument Clinic: Fix signature of optional positional-only arguments
Type: Stage:
Components: Argument Clinic Versions: Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: haypo, larry, rhettinger, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2017-01-17 15:17 by haypo, last changed 2017-01-17 15:52 by serhiy.storchaka.

File name Uploaded Description Edit
ac_optional_positional.patch haypo, 2017-01-17 15:17
getattr_ac.patch haypo, 2017-01-17 15:25
Messages (6)
msg285650 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-17 15:17
When a function has only positional arguments and at least one argument is optional, the expected signature is:

  func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]])

For example, the signature of format() is inconsistent with its documentation.

$ python3 -c 'help(format)'|cat
Help on built-in function format in module builtins:

format(value, format_spec='', /)
    Return value.__format__(format_spec)
    format_spec defaults to the empty string

.. function:: format(value[, format_spec])

Attached patch is a first attempt to fix the issue. The problem is that my heuristic to check if an argument is "optional" doesn't seem to work as expected in all cases. I chose to check if the C default is NULL.

The problem is that some functions defines a C default to NULL whereas the Python default is set to a different value and is correct.

Example with _io.FileIO.truncate:

    /*[clinic input]
        size as posobj: object = NULL

whereas the documentation says that the default is None:

   .. method:: truncate(size=None)

It's easy to fix the default, but in this case my change doesn't fix the signature anymore since the C default is still NULL:

    /*[clinic input]
        size as posobj: object(c_default="NULL") = None

We need a different heuristic than C default is NULL, or we should fix functions where the heuristic fails.
msg285651 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-17 15:18
List of functions modified when Argument Clinic is run to regenerate .c.h files:

msg285656 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-17 15:21
See also issue #20291: "Argument Clinic should understand *args and **kwargs parameters".
msg285657 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-17 15:25
This issue is blocking me to convert more functions to Argument Clinic. See for example attached getattr_ac.patch which converts getattr() to AC. Without ac_optional_positional.patch, AC generates the signature:

   "getattr($module, object, name, default=None, /)\n"

whereas the following signature is expected:

   "getattr($module, object, name[, default])\n"
msg285659 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-17 15:34
The problem is that

  func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]])

is not compatible with the inspect module.

In many case a meaningful default value was added if this is possible. For example the Python default shown in the signature can be set to '', 'utf-8' or 'strict' while the C default value is NULL for performance. If the parameter is upper index in the sequence it can be set to sys.maxsize (Py_SSIZE_T_MAX in C).

This is not always possible. For example there is not default value for dict.pop().
msg285661 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-17 15:52
Please don't change this part of Argument Clinic without Larry. There were several attempts to solve this problem, I don't like current status, but this is Larry's decision.
Date User Action Args
2017-01-17 15:52:04serhiy.storchakasetmessages: + msg285661
2017-01-17 15:34:50serhiy.storchakasetnosy: + yselivanov
messages: + msg285659
2017-01-17 15:25:17hayposetfiles: + getattr_ac.patch

messages: + msg285657
2017-01-17 15:21:44hayposetmessages: + msg285656
2017-01-17 15:18:15hayposetmessages: + msg285651
2017-01-17 15:17:14haypocreate