classification
Title: Argument Clinic: Fix signature of optional positional-only arguments
Type: Stage:
Components: Argument Clinic Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: haypo, larry, rhettinger, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2017-01-17 15:17 by haypo, last changed 2017-01-17 15:52 by serhiy.storchaka.

Files
File name Uploaded Description Edit
ac_optional_positional.patch haypo, 2017-01-17 15:17
getattr_ac.patch haypo, 2017-01-17 15:25
Messages (6)
msg285650 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-17 15:17
When a function has only positional arguments and at least one argument is optional, the expected signature is:

  func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]])

For example, the signature of format() is inconsistent with its documentation.

Signature:
---
$ python3 -c 'help(format)'|cat
Help on built-in function format in module builtins:

format(value, format_spec='', /)
    Return value.__format__(format_spec)
    
    format_spec defaults to the empty string
---

Documentation:
---
.. function:: format(value[, format_spec])
---

Attached patch is a first attempt to fix the issue. The problem is that my heuristic to check if an argument is "optional" doesn't seem to work as expected in all cases. I chose to check if the C default is NULL.

The problem is that some functions defines a C default to NULL whereas the Python default is set to a different value and is correct.

Example with _io.FileIO.truncate:

    /*[clinic input]
    _io.FileIO.truncate
        size as posobj: object = NULL
        /

whereas the documentation says that the default is None:

   .. method:: truncate(size=None)

It's easy to fix the default, but in this case my change doesn't fix the signature anymore since the C default is still NULL:

    /*[clinic input]
    _io.FileIO.truncate
        size as posobj: object(c_default="NULL") = None
        /

We need a different heuristic than C default is NULL, or we should fix functions where the heuristic fails.
msg285651 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-17 15:18
List of functions modified when Argument Clinic is run to regenerate .c.h files:

builtin_format
builtin_getattr
builtin_input
builtin_sum
cmath_log
_codecs_ascii_decode
_codecs_ascii_encode
_codecs_charmap_decode
_codecs_charmap_encode
_codecs_code_page_encode
_codecs_escape_decode
_codecs_escape_encode
_codecs_latin_1_decode
_codecs_latin_1_encode
_codecs_mbcs_encode
_codecs_oem_encode
_codecs_raw_unicode_escape_decode
_codecs_raw_unicode_escape_encode
_codecs_readbuffer_encode
_codecs_unicode_escape_decode
_codecs_unicode_escape_encode
_codecs_unicode_internal_decode
_codecs_unicode_internal_encode
_codecs_utf_16_be_encode
_codecs_utf_16_le_encode
_codecs_utf_32_be_encode
_codecs_utf_32_le_encode
_codecs_utf_7_encode
_codecs_utf_8_encode
_dbm_dbm_get
_dbm_dbm_setdefault
fcntl_fcntl
_imp_create_dynamic
_io_FileIO_truncate
os_putenv
os_unsetenv
pyexpat_xmlparser_ExternalEntityParserCreate
_sre_SRE_Match_end
_sre_SRE_Match_span
_sre_SRE_Match_start
_tkinter_create
unicodedata_UCD_decimal
unicodedata_UCD_digit
unicodedata_UCD_name
unicodedata_UCD_numeric
unicode_lstrip
unicode_maketrans
unicode_rstrip
msg285656 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-17 15:21
See also issue #20291: "Argument Clinic should understand *args and **kwargs parameters".
msg285657 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-17 15:25
This issue is blocking me to convert more functions to Argument Clinic. See for example attached getattr_ac.patch which converts getattr() to AC. Without ac_optional_positional.patch, AC generates the signature:

   "getattr($module, object, name, default=None, /)\n"

whereas the following signature is expected:

   "getattr($module, object, name[, default])\n"
msg285659 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-17 15:34
The problem is that

  func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]])

is not compatible with the inspect module.

In many case a meaningful default value was added if this is possible. For example the Python default shown in the signature can be set to '', 'utf-8' or 'strict' while the C default value is NULL for performance. If the parameter is upper index in the sequence it can be set to sys.maxsize (Py_SSIZE_T_MAX in C).

This is not always possible. For example there is not default value for dict.pop().
msg285661 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-17 15:52
Please don't change this part of Argument Clinic without Larry. There were several attempts to solve this problem, I don't like current status, but this is Larry's decision.
History
Date User Action Args
2017-01-17 15:52:04serhiy.storchakasetmessages: + msg285661
2017-01-17 15:34:50serhiy.storchakasetnosy: + yselivanov
messages: + msg285659
2017-01-17 15:25:17hayposetfiles: + getattr_ac.patch

messages: + msg285657
2017-01-17 15:21:44hayposetmessages: + msg285656
2017-01-17 15:18:15hayposetmessages: + msg285651
2017-01-17 15:17:14haypocreate