classification
Title: Incorrect __text_signature__ for sorted
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: 26282 Superseder:
Assigned To: serhiy.storchaka Nosy List: eriknw, martin.panter, ncoghlan, python-dev, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-04-10 17:54 by eriknw, last changed 2017-01-23 10:31 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
sorted_1.diff eriknw, 2016-04-10 17:54 Update the docstring of sorted builtin review
sorted_2.patch eriknw, 2016-04-12 05:37 flexible call signature; sorted matches original __text_signature__ review
sorted_3.patch eriknw, 2016-04-12 13:16 Correct __text_signature__ for sorted. Behavior unchanged. review
Messages (13)
msg263145 - (view) Author: Erik Welch (eriknw) * Date: 2016-04-10 17:54
The first argument to sorted is positional-only, so the text signature should be:

sorted($module, iterable, /, key=None, reverse=False)

instead of

sorted($module, iterable, key=None, reverse=False)

To reproduce the issue, attempt to use "iterable" as a keyword argument:

>>> import inspect
>>> sig = inspect.signature(sorted)
>>> sig.bind(iterable=[])  # should raise, but doesn't
>>> sorted(iterable=[])  # raises TypeError
msg263155 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-04-10 22:46
This is a strange case. It looks like “iterable” is half-supported as a keyword argument. So Silent Ghost’s patch fixes the signature, but the code still tries to accept keyword arguments:

>>> sorted(iterable=None)
TypeError: 'NoneType' object is not iterable
>>> sorted(iterable=())
TypeError: 'iterable' is an invalid keyword argument for this function

The problem is that sorted() blindly passes the keyword arguments to list.sort(). I guess we could delete "iterable" from them, but maybe it is not worth the trouble.
msg263222 - (view) Author: Erik Welch (eriknw) * Date: 2016-04-12 05:36
Interesting observation, Martin.

Upon further consideration, the call signature for sorted really is quite odd.  It doesn't behave like any other builtin function.  Currently, "iterable" is positional-only, and "key=" and "reverse=" are keyword only.  I would only expect such behavior for functions with variadic *args.

I uploaded a new patch so that the call signature matches the original __text_signature__.  This means "iterable" may be given as a keyword argument, and "key" and "reverse" may be given as positional arguments.

I added tests for the new behavior, and all tests pass for me.
msg263227 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-04-12 05:50
I don't think we should start down the path of changing APIs just to accommodate weakness in the generation of the text signature.

We don't want to encourage unreadable oddities like sorted(reverse=False, iterable=source).  Best readability comes from the required positional argument for the iterable.  Please don't create a new and unnecessary keyword argument.   The current API is intentional.
msg263230 - (view) Author: Erik Welch (eriknw) * Date: 2016-04-12 06:16
That's a fair and valid point, Raymond.  "sorted_2.patch" was submitted for consideration.  Either __text_signature__ is wrong, or the call argument handling is wrong.  One should be fixed.

Having a flexible call signature as if sorted were a user-defined function, such as "def sorted(iterable, key=None, reverse=False):", does allow for programmatic use of the introspected signature.  Here, using "iterable=" as a keyword can be convenient.

"sorted_1.diff" is wrong.  To match the existing call signature, __text_signature__ should be:

sorted($module, iterable, /, *, key=None, reverse=False)

I don't know any other builtin with a signature like this.  Such a signature may be a point of confusion for people learning Python ("what are those funny symbols?!"), who often encounter sorted early on.
msg263231 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-04-12 06:20
See issue26282.
msg263238 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-04-12 08:42
+1 for Serhiy's suggestion of enhancing the C API to specifically handle these cases.
msg263244 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-04-12 12:37
I didn’t realize about the keyword-only parameters. This is inherited from list.sort(). The signature can be corrected in 3.5.

There is also Issue 21314 about documenting the slash notation for signatures (it comes from PEP 457).
msg263246 - (view) Author: Erik Welch (eriknw) * Date: 2016-04-12 13:16
sorted_3.patch corrects the __text_signature__.  Behavior of sorted is unchanged.

>>> def raises(err, lamda):
...     try:
...         lamda()
...         return False
...     except err:
...         return True
...
>>> import inspect
>>> sig = inspect.signature(sorted) 
>>> # `iterable` is positional-only
>>> assert raises(TypeError, lambda: sorted(iterable=[]))
>>> assert raises(TypeError, lambda: sig.bind(iterable=[]))
>>> # `key` and `reverse` are keyword-only
>>> assert raises(TypeError, lambda: sorted([], lambda x: x))
>>> assert raises(TypeError, lambda: sig.bind([], lambda x: x))
msg263280 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-04-12 22:21
Patch 3 looks okay to me.
msg285923 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-20 21:10
See also issue29327 and issue29331.

sorted_3.patch LGTM.
msg286063 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-01-23 09:07
Serhiy, do you want to apply this?
msg286069 - (view) Author: Roundup Robot (python-dev) Date: 2017-01-23 10:31
New changeset c0a9fb3e19b9 by Serhiy Storchaka in branch '3.5':
Issue #26729: Fixed __text_signature__ for sorted().
https://hg.python.org/cpython/rev/c0a9fb3e19b9

New changeset fcb19fb42058 by Serhiy Storchaka in branch '3.6':
Issue #26729: Fixed __text_signature__ for sorted().
https://hg.python.org/cpython/rev/fcb19fb42058

New changeset 4ad195d9e184 by Serhiy Storchaka in branch 'default':
Issue #26729: Fixed __text_signature__ for sorted().
https://hg.python.org/cpython/rev/4ad195d9e184
History
Date User Action Args
2017-01-23 10:31:57serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: commit review -> resolved
2017-01-23 10:31:20python-devsetnosy: + python-dev
messages: + msg286069
2017-01-23 09:07:48rhettingersetmessages: + msg286063
2017-01-23 08:57:04rhettingersetassignee: rhettinger -> serhiy.storchaka
2017-01-20 21:10:31serhiy.storchakasetstage: patch review -> commit review
messages: + msg285923
versions: + Python 3.7
2016-04-12 22:21:38martin.pantersetmessages: + msg263280
2016-04-12 13:16:17eriknwsetfiles: + sorted_3.patch

messages: + msg263246
2016-04-12 12:37:07martin.pantersetmessages: + msg263244
2016-04-12 08:42:18ncoghlansetdependencies: + Add support for partial keyword arguments in extension functions
messages: + msg263238
2016-04-12 06:20:24serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg263231
2016-04-12 06:16:37eriknwsetmessages: + msg263230
2016-04-12 05:50:04rhettingersetassignee: rhettinger

messages: + msg263227
nosy: + rhettinger
2016-04-12 05:37:01eriknwsetfiles: + sorted_2.patch

messages: + msg263222
2016-04-10 22:46:59martin.pantersetnosy: + martin.panter
messages: + msg263155
2016-04-10 18:49:01SilentGhostsetnosy: + ncoghlan
stage: patch review

components: + Interpreter Core, - Extension Modules, Library (Lib)
versions: + Python 3.6
2016-04-10 17:54:40eriknwcreate