classification
Title: Document magic methods called by built-in functions
Type: enhancement Stage: patch review
Components: Documentation Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: andymaier, belopolsky, berker.peksag, cvrebert, docs@python, eric.araujo, mark.dickinson, martin.panter, rhettinger, serhiy.storchaka
Priority: low Keywords: patch

Created on 2010-11-01 23:54 by eric.araujo, last changed 2016-04-10 00:55 by rhettinger.

Files
File name Uploaded Description Edit
issue10289-magic-py34_v1.diff andymaier, 2014-07-07 10:31 Patch v1 for Python 3.4 and merging into default.
issue10289-magic-py34_v2.diff andymaier, 2014-07-07 10:50 Patch v2 for Python 3.4 and merging into default. review
Messages (11)
msg120189 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-01 23:54
At the top of Doc/library/functions.rst, which documents built-in functions like abs, getattr or hash, a comment reads “document all delegations to __special__ methods”.

Some functions are already good: enumerate for instance does link to the definition of iterator and hints about the __next__ method, format points to __format__, etc.  They can serve as example for how to add links (in plain text and in the global index).
msg120191 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-11-02 00:03
Éric,

I just wanted to link to a related discussion we had under issue 8983.  See msg107689.
msg222290 - (view) Author: Andy Maier (andymaier) * Date: 2014-07-04 15:26
I have reviewed the descriptions of the built-in functions in Python 3.4, and found only the following issues w.r.t. missing __special__functions:

1. getattr(), setattr(), delattr(): They only refer to object attributes and miss to mention the fallback to object.__getattr__(), etc. Because hasattr() calls getattr() to test for the presence, the __getattr__() is relevant for hasattr() as well.

2. len() misses to describe that it uses s.__len__().

3. sorted() misses to describe how rich comparison methods can be used. I think we can integrate the changes to list.sort() proposed in issue14050.

4. str() delegates the description to stdtypes.html#str, which in turn does describe obj.__str__(). Not sure we need to do something for str().

I did not check 2.7 yet.

Andy
msg222443 - (view) Author: Andy Maier (andymaier) * Date: 2014-07-07 10:31
Uploaded a patch for Python 3.4, and for merging into "default".

The patch addresses items 1) to 3) from my previous post; item 4) does not need to be addressed IMHO.

Andy
msg222446 - (view) Author: Andy Maier (andymaier) * Date: 2014-07-07 10:50
Uploaded v2 of the 3.4/default patch, which removes the comment line at the top of Doc/library/functions.rst (mentioned by Éric in the original message of this issue).

-> Please review the patch.

-> Please also double check whether there are any additional built-in functions in 3.x that have a need to document their underlying __special__ functions.

Andy
msg222477 - (view) Author: Andy Maier (andymaier) * Date: 2014-07-07 17:07
Comments on the patch py34_v2:

1. On complex(): It delegates to object.__complex__(); that should also be described.

2. On hex(): The use of "__index__()" is text and should be changed to a hyperlink.

Andy
msg222480 - (view) Author: Andy Maier (andymaier) * Date: 2014-07-07 17:16
I reviewed the description of the built-in functions in Python 2.7, and found these issues:

1. The following built-in functions do not mention their underyling __special__ functions:
- cmp()
- delattr(), getattr(), hasattr(), setattr()
- complex(), int(), long(), float()
- hash()
- len()
- str()
- int(), oct()

2. The following built-in functions list the __special__ functions they invoke as text that is not a hyperlink:
- dir()
- format()
- hex()

3. The following built-in functions could be improved from the description of the 3.x version:
- bool(): References and links from 3.x version could be added.
- bytearray(): References from last line of 3.x version could be added.
- callable(): Wording "if they have a..." could be improved by using 3.x wording.

4. The codepage of the ASCII string for chr() is not documented.

Andy
msg222486 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-07-07 18:05
Should the Python 2.7 delegation of `round` to `__float__` for its input also be described as part of this issue?

>>> class A(object):
...     def __float__(self): return 1729.1
... 
>>> round(A())
1729.0
msg222487 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-07 18:13
Could #15436 "__sizeof__ is not documented" also be covered by this?
msg240517 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-12 00:49
Sizeof is not relevant to the built-in functions as far as I know. It is already described at <https://docs.python.org/dev/library/sys.html#sys.getsizeof>.

Some more delegations that I think should be documented here (at least for Python 3):

* abs() -> object.__abs__()
* bytes() -> object.__bytes__()
* complex() -> object.__complex__()
* divmod() -> object.__divmod__()
* float() -> object.__float__(). The text is already there, but there is no hyperlink.
* hex() -> object.__index__(). Also just needs a hyperlink.
* isinstance() -> class.__instancecheck__()
* issubclass() -> class.__subclasscheck__()
* pow() -> object.__pow__()
* round() -> object.__round__()

I added some comments about the sorted() specification on Reitveld, but maybe they should be handled in a separate issue, because the delegation is less direct compared to the other functions being considered here. And any update to sorted() should probably also consider list.sort(), min(), max(), and maybe the bisect and heapq modules.
msg263119 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-04-10 00:54
Most of the proposed update look reasonable updates and would improve the documentation.  

That said, please take care to not accidentally document and unintentionally guarantee implementation details rather than language requirements (leaving freedom for future changes to implementation and freedom for IronPython, PyPy, and Jython to use their best possible implementations).

Sizeof is a CPython specific implementation detail.

I also have reservations about class.__instancecheck__() and class.__subclasscheck__() which are more appropriately described in a section on abstract base classes than for the otherwise simple and clear docs for isinstance() and issubclass().

The sort() method does guarantee use of __lt__ but other tools that make comparisons make or may not follow that lead (i.e. heapq used to use __le__ and collections.abc.Set uses both __le__ and __ge__).  Accordingly, there is a PEP 8 recommendation to use @functools.total_ordering rather than supplying just a single rich comparison method).

One other thought is to keep the additions as brief as possible to not distract from the main message of each section; keeping the docs primarily focused on what a function does rather than how it does it, and remembering that making the docs more lengthly impairs their utility for everyday use.  Our docs are already much more chatty than equivalents in Java and Go (it used to take five minutes to read the docs for the builtin functions and now it takes an hour). 

There is also a matter of keeping the docs approachable for normal people.  For most folks, saying that hex(num) returns a string with the number in hexadecimal would suffice.  Adding notes about the exact case of the letters, handling of negatives, use of __int__ and __index__, and comparisons with string formatting, and multiple examples beats a dead horse and makes a simple tool seem more complex.

Though I think this so go forward, I'm marking it with "low" priority because we don't have any evidence that users have found the current docs to be inadequate -- they have served well over Python's long history.   Cases we've gone into details like __format__ or __next__ were there because they we essential to the tool; in contrast, it is unlikely that a user would ever need to know the sys.getsizeof() delegated its work to __sizeof__.
History
Date User Action Args
2016-04-10 00:55:02rhettingersetpriority: normal -> low
nosy: + rhettinger
messages: + msg263119

2016-04-09 13:01:49BreamoreBoysetnosy: - BreamoreBoy
2016-04-09 12:02:31serhiy.storchakasetnosy: + serhiy.storchaka

versions: + Python 3.6, - Python 3.4
2015-04-12 08:31:34berker.peksagsetnosy: + berker.peksag

type: enhancement
stage: needs patch -> patch review
2015-04-12 00:49:50martin.pantersetnosy: + martin.panter
messages: + msg240517
2014-07-13 21:47:06cvrebertsetnosy: + cvrebert
2014-07-07 18:13:11BreamoreBoysetnosy: + BreamoreBoy
messages: + msg222487
2014-07-07 18:05:31mark.dickinsonsetnosy: + mark.dickinson
messages: + msg222486
2014-07-07 17:16:51andymaiersetmessages: + msg222480
2014-07-07 17:07:41andymaiersetmessages: + msg222477
2014-07-07 10:50:04andymaiersetfiles: + issue10289-magic-py34_v2.diff

messages: + msg222446
2014-07-07 10:31:31andymaiersetfiles: + issue10289-magic-py34_v1.diff
keywords: + patch
messages: + msg222443
2014-07-04 15:26:17andymaiersetnosy: + andymaier

messages: + msg222290
versions: + Python 3.4, Python 3.5, - Python 3.1, Python 3.2
2010-11-02 00:03:09belopolskysetnosy: + belopolsky
messages: + msg120191
2010-11-01 23:54:55eric.araujocreate