classification
Title: Doc that collection mutation methods return item or None
Type: Stage: committed/rejected
Components: Documentation Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: cvrebert, docs@python, ezio.melotti, mikehoy, python-dev, rhettinger, stutzbach, terry.reedy
Priority: normal Keywords: easy, patch

Created on 2011-05-26 21:33 by terry.reedy, last changed 2011-10-08 16:31 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
return-none.diff mikehoy, 2011-10-08 16:02 Return None added in Docstrings and Documentation for list.append(), list.remove(), list.sort(), and list.extend() review
Messages (7)
msg137009 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-26 21:33
BACKGROUND
One of most common recurring topics on python-list (perhaps monthly) is newbies tripping over a mutation method returning None when they expect the collection. Today's example: "Puzzled by list-appending behavior".
An excerpt from the responses: 

"On 5/26/2011 11:58 AM, MRAB wrote:
> On 26/05/2011 06:17, Chris Rebert wrote:

>> list.remove(), list.sort(), and list.extend() similarly return None
>> rather than the now-modified list.

> I'd just like to point out that it's a convention, not a rigid rule.
> Sometimes it's not followed, for example, dict.setdefault."

In Python 1, I believe it was true that all mutation methods of mutable builtin collections (ie, lists -- I do not think dicts had any and sets did not exist yet) returned None. With the introduction of list.pop, the return-None rule was 'broken', though the meme continues. However, the flip side of that rule, do-not-return-the-collection, continues to be true. And the return_None rule was not really broken, but broadened to "return None or an item from the collection". I believe this should be better documented than it is now.

PROPOSALS

1. State the general rule.

A. Tutorial: At the top of chapter 5, just after "This chapter describes some things you’ve learned about already in more detail, and adds some new things as well.", add something like

"For lists, sets, and dicts, methods that change the contents or order never return the instance. Instead, they return an item from the instance or, more commonly, None."

B. Library Manual: Near the top of 4. Built-in Types, after "The principal built-in types are numerics, sequences, mappings, classes, instances and exceptions.", add something like

"Some collection classes are mutable. The methods that add, subtract, or rearrange their members never return the collection instance itself but instead return an item from the instance or None."

Comment: This rule applies to special methods like __getitem__ and __setitem__. 'lis.append(item)' is equivalent to lis.__setitem__(len(lis):len(lis), item), so it should not be so surprising that it has the same return.

2. Document None returns explicitly.

They are currently documented implicitly, by absence.

A. Docstrings: Compare the relevant parts of the output from 'help(list.append)' and 'help(list.pop)'

   L.append(object) -- append object to end
   L.pop([index]) -> item -- remove and return item at index 

Add ' -> None' to specify return for .append.
   L.append(object) -> None -- append object to end

I expect experienced Python programmers may object that this puts too much emphasis on the unimportant null return, but it is important that Python programmers know about it and experience shows that many newbies *need* that emphasis.

B. Doc entries: Essentially same suggestion -- add 'Return None.' to appropriate entries in tutorial chapter 5. For library manual, possibly add footnote '(9) Returns None' to the method table in 4.6.4. Mutable Sequence Types. For sets and dicts, add 'and return None' to appropriate entries.
msg137029 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-05-27 07:13
+1 on the doc suggestions

-1 on any hope that casual users will have read or remembered them.  ISTM that a common theme among the post of people getting tripped-up by this is that they aren't doing more than a quick skim of the docs anyway.
msg137041 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-05-27 08:52
I find this wording a little confusing: "For lists, sets, and dicts, methods that change the contents or order never return the instance. Instead, they return an item from the instance or, more commonly, None.".
I would suggest to drop the second part because we don't care about those methods now and because some of them return None too, and simply use "Methods that mutate the object never return the instance.", possibly adding something like "to remind you that no new objects have been created.".
The same wording can be used in both the Tutorial and the Library Manual (if readers happen to read it twice, it's even more likely that they will remember it).

FWIW the glossary has entries for "mutable" and "immutable", adding a note about this convention in the "mutable" entry might be a good idea too.

Adding the '-> None' in the docstring is fine too, I'm not sure about the footnote.

Terry, do you want to work on a patch?
msg137083 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2011-05-27 16:45
> Comment: This rule applies to special methods like __getitem__
> and __setitem__. 'lis.append(item)' is equivalent to
> lis.__setitem__(len(lis):len(lis), item), so it should not be so
> surprising that it has the same return.

It's not true for all special methods.  __iadd__ and __imul__ return the list itself.

That's a minor nitpick though.  +1 on adding "and return None" to the docstrings of methods that often trip people up.  Not everyone will read them, but some people will.
msg137096 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-27 18:58
Darn, I knew there might be an exception I was overlooking ;-).
Anyway, I considered the general statements to be drafts, to be rewritten after comments.

As to the footnote (9) suggestion: the set and dict sections list each method separately with normal entries, so there is not no problem with adding 'and returns None' to those that need it. The mutable sequence section is unique in instead putting all the methods in a condensed table. 'return None' could be added to some lines in the table, but not others.
msg145174 - (view) Author: Mike Hoy (mikehoy) * Date: 2011-10-08 16:02
Created a patch based on suggestions here.
msg145176 - (view) Author: Roundup Robot (python-dev) Date: 2011-10-08 16:31
New changeset 352d075839f7 by Georg Brandl in branch 'default':
Closes #12192: Document that mutating list methods do not return the instance (original patch by Mike Hoy).
http://hg.python.org/cpython/rev/352d075839f7
History
Date User Action Args
2011-10-08 16:31:04python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg145176

resolution: fixed
stage: needs patch -> committed/rejected
2011-10-08 16:02:33mikehoysetfiles: + return-none.diff
keywords: + patch
messages: + msg145174
2011-10-07 14:06:15mikehoysetnosy: + mikehoy
2011-05-27 18:58:58terry.reedysetmessages: + msg137096
2011-05-27 16:45:36stutzbachsetnosy: + stutzbach
messages: + msg137083
2011-05-27 08:52:52ezio.melottisetnosy: + ezio.melotti
messages: + msg137041

keywords: + easy
stage: needs patch
2011-05-27 07:13:38rhettingersetnosy: + rhettinger
messages: + msg137029
2011-05-26 22:31:59cvrebertsetnosy: + cvrebert
2011-05-26 21:33:10terry.reedycreate