classification
Title: Docs: Clarify NotImplemented use cases
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.9
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: docs@python Nosy List: aeros, docs@python, eric.araujo, ezio.melotti, jdemeyer, mdk, rhettinger, veky, willingc
Priority: normal Keywords: easy

Created on 2019-08-23 21:49 by aeros, last changed 2019-09-05 08:47 by veky. This issue is now closed.

Messages (11)
msg350333 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-08-23 21:49
In the documentation for the NotImplemented constant (https://docs.python.org/3/library/constants.html#NotImplemented), the only use case mentioned is for binary special methods, (such as object.__eq__(other)). However, based on a conversation in a recent PR (https://github.com/python/cpython/pull/15327#discussion_r316561140), there seems be several other use cases as well.  It's quite useful to utilize when it's desired to express that a particular operation is not supported, without raising an error.

Expanding upon the use cases will provide readers an idea of other cases in which the constant could be useful. Also, the current wording could potentially come across as implying that the _only_ use case for the constant is for binary special methods, and as far as I'm aware, that is not the case.
msg350481 - (view) Author: Vedran Čačić (veky) * Date: 2019-08-26 03:47
Well, it is the _intended_ use of NotImplemented. Of course you can use it whenever you want, just as you can use Ellipsis whenever you want. But I don't think docs should encourage that.
msg350482 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-26 04:26
It is not the purpose of the docs to list use cases.  Mostly we say what something does or how it is defined.  As Vedran says, how people use it is their own business.

Also, please be careful expanded the docs.  They quickly become a promise.  Per Doc/reference/introduction.rst, "It is dangerous to add too many implementation details to a language reference document --- the implementation may change, and other implementations of the
same language may work differently."
msg350969 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-09-02 05:27
Thanks for the feedback Vedran and Raymond.

> It is not the purpose of the docs to list use cases.  Mostly we say what something does or how it is defined. As Vedran says, how people use it is their own business.

The underlying issue here seems to be that I misunderstood the existing section to suggest to suggest binary special methods are the only use case, which is probably a good argument in favor of not listing additional use cases. This can be misinterpreted as suggesting that others are not recommended, and lead to further confusion.


First sentence of the NotImplemented documentation:

> Special value which should be returned by the binary special methods (e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods (e.g. __imul__(), __iand__(), etc.) for the same purpose. 

Would it be viable to rephrase the existing section in a manner that explains the functional purpose of NotImplemented without revolving around its use case in binary special methods?

> Also, please be careful expanded the docs.  They quickly become a promise.

Good point, I may not have adequately considered that mentioning a use case turns into a promise for that functionality. This could easily result in a significant long-term maintenance cost.
msg350970 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-09-02 05:45
> Would it be viable to rephrase the existing section in a manner that explains the functional purpose of NotImplemented without revolving around its use case in binary special methods?

To expand further upon this, here's an initial idea for improving the first sentence:

A special value used to indicate that an operation is not supported between specific types.

The section regarding it's usage in binary special methods could potentially remain. I'm thinking the main issue here (if there is one) is that the NotImplemented constant is defined _exclusively_ from a specific use case, rather than its general purpose.
msg351005 - (view) Author: Vedran Čačić (veky) * Date: 2019-09-02 14:24
Sorry, I think you still don't understand.

The emulation of double dispatch by single dispatch, with all complications it brings, is the only reason NotImplemented exists. If Python didn't have binary operators (or inheritance), I'm quite sure it wouldn't have NotImplemented. The idea was never "oh, this method isn't supported", but much more precise "this half of protocol, with given ordering of operands according to a class hierarchy, isn't implemented - try the other half".

Of course, you might argue that _once Python has NotImplemented_, it can be used elsewhere - but as I said, I don't think it should be encouraged.
msg351055 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-09-03 04:38
Thanks for the explanation.

> Of course, you might argue that _once Python has NotImplemented_, it can be used elsewhere - but as I said, I don't think it should be encouraged.

Hmm, okay. My understanding of Raymond's explanation was more so "let's not encourage this because we don't want to guarantee the behavior" rather than "using it outside of binary operators shoudn't be encouraged".

Prior to Jerome's usage of it in ``fractions._as_integer_ratio()`` (https://github.com/python/cpython/pull/15327/files), I had not seen it used outside of special binary operators. I thought it was an interesting usage of NotImplemented, and the current phrasing of the documentation seemed to implicitly discourage it from being used in that manner.
msg351056 - (view) Author: Vedran Čačić (veky) * Date: 2019-09-03 04:50
Of course, languages evolve. Annotations were introduced just for function arguments, now we use them almost everywhere. Generators were simply loop managers, now they've blown up to complete asynchronous programming beasts. Ellipsis was introduced for easier slicing of n-dimensional objects, now we have it all over the language - precisely because it's so versatile. NotImplemented, I _think_ (but I'm really no expert), is not.

As you say, we currently have only one usage of NotImplemented outside its intended purpose. Maybe we should wait to see whether it becomes at least a little bit more popular, before thinking about blessing it.

About Raymond's post: well, there can be orthogonal reasons why something isn't a good idea. That doesn't mean it _is_ a good idea. :-]
msg351148 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-09-04 19:03
> As you say, we currently have only one usage of NotImplemented outside its intended purpose.

I know at least 3 in CPython, so it's not so rare to use NotImplemented for something else than binary operators:
1. __subclasshook__
2. reducer_override (in pickling)
3. __length_hint__

> Of course, you might argue that _once Python has NotImplemented_, it can be used elsewhere - but as I said, I don't think it should be encouraged.

I'm not saying that it should be actively encouraged, but the documentation shouldn't be limited to just one use case. Given that NotImplemented exists, why shouldn't it be used in more cases to indicate that an operation is not implemented?
msg351181 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-09-05 06:46
> As you say, we currently have only one usage of NotImplemented outside its intended purpose. Maybe we should wait to see whether it becomes at least a little bit more popular, before thinking about blessing it.

> I know at least 3 in CPython, so it's not so rare to use NotImplemented for something else than binary operators


I'm thinking that it might be worthwhile to start a discussion about this on python-dev, to see what the others think about how the NotImplemented docs should be defined, and whether or not it should be used outside of binary operators.

Based on the feedback from Raymond and Vedran, I'm in agreement that it might not be the best idea in the long term to keep adding additional use cases to the documentation. But, it really doesn't make much sense to me that the constant is entirely defined around a single use case in the doc while we're using it for other purposes throughout CPython.
msg351192 - (view) Author: Vedran Čačić (veky) * Date: 2019-09-05 08:47
Of course, if there are independent use cases already in the codebase, then my opinion is changed.
History
Date User Action Args
2019-09-05 08:47:39vekysetmessages: + msg351192
2019-09-05 06:46:22aerossetmessages: + msg351181
2019-09-04 19:03:26jdemeyersetmessages: + msg351148
2019-09-03 04:50:37vekysetmessages: + msg351056
2019-09-03 04:38:48aerossetmessages: + msg351055
2019-09-02 14:24:34vekysetmessages: + msg351005
2019-09-02 05:45:05aerossetmessages: + msg350970
2019-09-02 05:27:22aerossetmessages: + msg350969
2019-08-26 04:26:49rhettingersetstatus: open -> closed

nosy: + rhettinger
messages: + msg350482

resolution: rejected
stage: needs patch -> resolved
2019-08-26 03:47:02vekysetnosy: + veky
messages: + msg350481
2019-08-24 08:56:28jdemeyersetnosy: + jdemeyer
2019-08-23 21:49:06aeroscreate