classification
Title: Inconsistency in documentation of operator.index
Type: enhancement Stage: needs patch
Components: Documentation Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, madphysicist, r.david.murray, rhettinger, terry.reedy
Priority: low Keywords:

Created on 2017-07-26 03:02 by madphysicist, last changed 2017-07-29 01:59 by terry.reedy.

Messages (5)
msg299195 - (view) Author: Joseph Fox-Rabinovitz (madphysicist) * Date: 2017-07-26 03:02
The docs for [`operator.index`][1] and `operator.__index__` state that

> Return *a* converted to an integer. Equivalent to `a.__index__()`.

The first sentence is correct, but the second is not. First of all, we have the data model [docs][2]:

> For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.

Secondly, we can make a simple counter-example in code:

```
import operator

class A:
    def __index__(self): return 0

a = A()
a.__index__ = (lambda self: 1).__get__(a, type(a))
operator.index(a)
```

The result is of course zero and not one.

I believe that the docs should read something more like one of the following to avoid being misleading:

> Return *a* converted to an integer, if it is already an integral type.

> Return *a* converted to an integer. Equivalent to `type(a).__index__(a)`.

Or a combination of both:

> Return *a* converted to an integer, if it is already an integral type. Equivalent to `type(a).__index__(a)`.

  [1]: https://docs.python.org/3/library/operator.html#operator.index
  [2]: https://docs.python.org/3/reference/datamodel.html#special-method-lookup
msg299340 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-07-27 19:44
This seems like a generic issue for magic methods and is already covered by "for custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary."

While you're technically correct with suggesting "Equivalent to `type(a).__index__(a)`", I don't think this is an improvement.  It makes the docs safe against overly pedantic readings, but it also reduces the intelligibility for everyday users.

The usual approach in the docs is to say "a[b] <==> a.__getitem__(b)" rather than "a[b] <==> type(a).__getitem__(a, b)".  The latter is more correct but it is also less helpful.  For the most part, this style of presentation has worked well for a lot of people for a long time.

I recommend closing this or not doing any more than changing "Equivalent to:" to "Roughly equivalent to:".
msg299344 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-07-27 20:04
I agree with Raymond.  I'm not sure that adding roughly is going to decrease the possibility of confusion, but I won't object to it.

In a way, it's too bad we didn't make the attribute lookup machinery look up all dunder methods on the class, so that a.__index__ would call the class method.  I think backward compatibility prevented that.
msg299345 - (view) Author: Joseph Fox-Rabinovitz (madphysicist) * Date: 2017-07-27 20:09
I brought up the issue because it was really a point of confusion for me.
Could we make the change to "Roughly equivalent" and make that a link to
https://docs.python.org/3/reference/datamodel.html#special-method-lookup?
That would make it clear how the lookup is actually done.

While I agree that making the docs unnecessarily pedantic is probably a bad
thing, I am going to guess that I am not the only person that looks to them
for technical accuracy.

Regards,

    -Joe

On Thu, Jul 27, 2017 at 4:04 PM, R. David Murray <report@bugs.python.org>
wrote:

>
> R. David Murray added the comment:
>
> I agree with Raymond.  I'm not sure that adding roughly is going to
> decrease the possibility of confusion, but I won't object to it.
>
> In a way, it's too bad we didn't make the attribute lookup machinery look
> up all dunder methods on the class, so that a.__index__ would call the
> class method.  I think backward compatibility prevented that.
>
> ----------
> nosy: +r.david.murray
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue31042>
> _______________________________________
>
msg299445 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-29 01:59
To me, 'roughly' is wrong.  Either the equivalence is exact, or it is completely absent .  There is no 'nearly' or 'roughly' about this situation.

This is difference from  iterator_class_x(args) being mathematically equivalent to generator_function_y(args) in the sense of yielding *exactly* the same sequence of objects, but being different in the Python sense that type(iterator_class_x) != type(generator_function_y).  

Note: even in this case, I was once in favor of changing 'equivalent' to 'roughly equivalent' in the itertools doc.  I now regret that because 'roughly' could be misunderstood.  I think that 'mathematically equivalent' or 'equivalent when iterated' or 'equivalent*' would be better, with an explanatory note at the top.

As for this issue, __index__ is a reserved name.  https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers

a.__index__ = <whatever> is an unauthorized use of a *reserved* word and the effect of such usage is not and need not be documented.  

The entry for __*__ does include "*Any* use of __*__ names, in any context, that does not follow explicitly documented use, is subject to breakage without warning."  To me, that says that the effect of the reserved-word assignment is undefined.  It could be made to raise an exception.

To be even clearer, I believe we should explicitly state what I consider implicit: something like "Any such use breaks these manuals, in the sense that it may make statements herein untrue.  These manuals assume that reserved names are used as specified."
History
Date User Action Args
2017-07-29 01:59:19terry.reedysetversions: - Python 3.3, Python 3.4
nosy: + terry.reedy

messages: + msg299445

type: enhancement
stage: needs patch
2017-07-27 20:09:18madphysicistsetmessages: + msg299345
2017-07-27 20:04:34r.david.murraysetnosy: + r.david.murray
messages: + msg299344
2017-07-27 19:44:13rhettingersetpriority: normal -> low
nosy: + rhettinger
messages: + msg299340

2017-07-26 03:03:19madphysicistsettype: behavior -> (no value)
2017-07-26 03:02:59madphysicistcreate