classification
Title: FAQ need list mutation answers
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ezio.melotti Nosy List: BreamoreBoy, Fran.Bull, berker.peksag, docs@python, eric.araujo, ezio.melotti, francismb, m123orning, mvolz, python-dev, r.david.murray, rhettinger
Priority: normal Keywords: patch

Created on 2014-01-06 03:37 by m123orning, last changed 2014-09-29 14:26 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
20135.patch Fran.Bull, 2014-01-17 05:26 review
defaultdoc_listmut.patch mvolz, 2014-03-10 17:47 Patch for FAQ docs on default branch review
sortFAQ_defaultval.patch mvolz, 2014-03-24 14:41 V2 of patch: changed title of sort question, moved default value question review
issue20135.diff ezio.melotti, 2014-07-06 19:15
issue20135-2.diff ezio.melotti, 2014-08-03 13:26 review
mutable_faq.patch r.david.murray, 2014-09-27 22:31 review
Messages (24)
msg207410 - (view) Author: Xiaoqing Rong (m123orning) Date: 2014-01-06 03:37
I was using IDLE (Python GUI) for Python 3.3.3. I don't know if this is a bug or you guys intend it to be this way:

>>> x=[1,2]
>>> y=x
>>> y.append(3)
>>> x
[1, 2, 3]

personally i'd prefer x stays as [1,2] when i'm trying to mutate y

also:
>>> def f1(m,n=['haha']):
	if m==n:
		print('m==n')
	else:
		print('m!=n')
	n.append('yaya')
	
>>> f1(['haha'])
m==n
>>> f1(['haha'])
m!=n

I'd prefer getting consistent results when calling functions like f1
msg207411 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-06 03:53
Both of these are FAQs, but to my surprise there don't seem to be answers to them in the FAQ list.  We should add some.

Yes, both of these behaviors is intentional.  The first is an important part of the language design: 'variable' names are just pointers to objects, so x and y point to the same object.  The second is the same issue combined with another design feature: the objects on the right side of the = in a def statement are evaluated/created at the time the def statement is executed, not at the time the function is called.  So 'n' points to the same object every time the function is called.

I'm going to leave this open until someone either points me to the FAQ entries I missed, or we add them.
msg208297 - (view) Author: Fran Bull (Fran.Bull) Date: 2014-01-16 18:46
I read the FAQ last night and I couldn't see these answered there either. I would like to try submitting a patch for this one, probably this evening. It will likely be two FAQs in the programming section that go something like:

Why does changing one list change another different list?
This happens:
>>> a = [1, 2, 3]
>>> b = a
>>> b.append(4)
>>> print a
[1, 2, 3, 4]

because variables are just names for things, in this case 'a' is the list we first defined and then b = a says that 'b' is also a name for that list. They are both the same list.

Why are my default args wrong?
This happens:

>>> from datetime import datetime
>>> class A(object):
...   def __init__(self, created_time=datetime.now()):
...     self.created_time = created_time
... 
>>> an_a = A()
>>> another_a = A()
>>> an_a.created_time
datetime.datetime(2014, 1, 16, 10, 40, 54, 33283)
>>> another_a.created_time
datetime.datetime(2014, 1, 16, 10, 40, 54, 33283)

because default arguments are evaluated when they're read for the first time by the interpreter. Usually when the class is imported. A good way to get the above to do what you want is to use a default argument of None and check for it, like:
>>> class B(object):
...   def __init__(self, created_time=None):
...     if created_time is None:
...       created_time=datetime.now()
...     self.created_time = created_time
... 
>>> a_b = B()
>>> another_b = B()
>>> a_b.created_time
datetime.datetime(2014, 1, 16, 10, 44, 44, 956710)
>>> another_b.created_time
datetime.datetime(2014, 1, 16, 10, 44, 51, 71311)

Feedback appreciated, particularly I'm not sure if this:
'default arguments are evaluated when they're read for the first time by the interpreter'
is exactly the right language. I guess I'll look it up.
msg208305 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-16 20:25
Thanks for working on this.

Technically, defaults are evaluated when the 'def' statement is executed, which is normally, but not always, at the time that the module is first imported (counterexample: nested function definitions).

The answer should also explicitly cover the more common mistake made with default keyword values of passing a mutable object and having it get persistently modified,and the question phrasing should reflect that.  The datetime example is a great first example before getting to the more common one, though.
msg208312 - (view) Author: Fran Bull (Fran.Bull) Date: 2014-01-16 21:09
Perhaps a 3rd FAQ something like this?:

Why is changing a list in one instance of a class also changing it in another instance of the same class?
This happens:
>>> class A(object):
...   def __init__(self, fruit=[]):
...     self.fruit = fruit
... 
>>> an_A = A()
>>> an_A.fruit.append('apple')
>>> another_A = A()
>>> print another_A.fruit
['apple']
>>> another_A.fruit.append('banana')
>>> print another_A.fruit
['apple', 'banana']
>>> print an_A.fruit
['apple', 'banana']
>>> print an_A.fruit is another_A.fruit
True
>>> print an_A.fruit is A().fruit
True

because of a combination of the above two FAQs, first the default argument is evaluated when the 'def' statement is executed and creates an instance of list. After that new instances of A have a variable 'fruit' that is the name for that instance of list.

I'm not sure whether I should be using as general terms as possible for the Q, i.e. 'mutable object' instead of 'list'. I'll reread http://docs.python.org/devguide/documenting.html and the FAQs before writing the patch.
msg208337 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-01-17 14:27
http://docs.python.org/3/faq/design.html#why-are-default-values-shared-between-objects
msg208342 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-17 15:36
I *thought* there was a FAQ entry for that.  Didn't think to look for it in the design section though :(.  Nor is the title likely to catch the eye of someone wondering why their default argument modifications are being unexpectedly persistent.  Also, the fact that it starts out with the sentence "This type of bug commonly bites neophyte programmers" is, IMO, a clue it should be in the programming section.
msg213052 - (view) Author: M. Volz (mvolz) Date: 2014-03-10 17:47
I've written and attached a patch for the list mutation question. I took a little bit of a different tack on this and addressed a more general question whilst still including list mutation as an example. I know the language can get a little fussy on these sorts of things- let me know if there's anything that could be better worded. 

As for the second issue, as Ezio mentioned there's already an FAQ question regarding that- would there be any objection to just copying it into the programming section? Or should it be left where it is?
msg213057 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-03-10 18:21
Resources: http://nedbatchelder.com/text/names.html
           http://python.net/~mwh/hacks/objectthink.html
msg213073 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-10 19:18
I'm in favor of moving the FAQ answer Ezio identified to the Programming section, myself, but I'd like concurrence from at least one other committer before we do it.
msg213204 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-03-12 06:31
I agree; if someone goes to the docs, “Programming FAQ” looks more enticing than “Design and History”.
msg213277 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-12 19:02
M. Votz:  I like your entry, but I think it should be titled "Why did changing list 'a' also change list 'b'?"  That's the form I've always encountered this question in.  Then the answer can start of with "It didn't, 'a' and 'b' are the same list." and then go on with your text.

Since Éric concurs, lets go ahead and move that other answer.  Probably to right before "How can I pass optional or keywords parameters from one function to another?"
msg214688 - (view) Author: M. Volz (mvolz) Date: 2014-03-24 14:41
Thanks David,

I've updated the patch to move the default values question into the programming FAQ where you recommended it go, and also changed the title of the mutable list question.
msg222410 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-07-06 17:58
New changeset 3881c12fa3ae by Ezio Melotti in branch '2.7':
#20135: move FAQ about mutable default arguments to the programming FAQs page.
http://hg.python.org/cpython/rev/3881c12fa3ae

New changeset 3b7b0f5aac1e by Ezio Melotti in branch '3.4':
#20135: move FAQ about mutable default arguments to the programming FAQs page.
http://hg.python.org/cpython/rev/3b7b0f5aac1e

New changeset f2a36b01ac02 by Ezio Melotti in branch 'default':
#20135: merge with 3.4.
http://hg.python.org/cpython/rev/f2a36b01ac02
msg222412 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-07-06 19:15
I moved the FAQ about mutable default arguments to the programming FAQs page.  I was going to do a review about new FAQ, but it since I had several comments I just tried to rewrite it and make a new patch.
Do you think this is clear enough?
msg222413 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-06 19:34
+1 from me, I thought it was crystal clear.
msg222440 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2014-07-07 07:11
LGTM.
msg222784 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-07-11 19:48
I think the example would be clarified by speaking about mutation operations versus non-mutation operations.  After all:

>>> x = [1]
>>> y = x
>>> x = x + [2]
>>> x
[1, 2]
>>> y
[1]

At that point including a list += operation would also be beneficial.
msg222812 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-07-11 23:31
Good point, I'll try to add that to the FAQ.
msg224635 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-08-03 13:26
Here is a new patch that includes an additional paragraph about mutation and non-mutation operations.
msg224645 - (view) Author: Francis MB (francismb) * Date: 2014-08-03 16:31
A) On the example:

+Also note that some operations (e.g. ``y.append(10)``/``y += [10]`` or
+``y.sort()``) mutate

are you saying:
1) "y.append(10)" divided by "y += [10]" or
2) "y.append(10)" and "y += [10]"

I don't want to split hairs but reading it fast confused me a bit


B) The paragraph:

+  etc.), we can use some specific operations to mutate it and all the variables
+  that refer to it will see
+  the change;

Couldn't be (one newline less):

+  etc.), we can use some specific operations to mutate it and all the variables
+  that refer to it will see the change;
msg227741 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-09-27 22:31
Here is my version of Ezio's patch.
msg227798 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-09-29 14:24
New changeset 138f54622841 by R David Murray in branch '3.4':
#20135: FAQ entry for list mutation.  (See also 6375bf34fff6.)
https://hg.python.org/cpython/rev/138f54622841

New changeset 3d924bbfdcbc by R David Murray in branch 'default':
Merge: #20135: FAQ entry for list mutation.  (See also 90b07d422bd9.)
https://hg.python.org/cpython/rev/3d924bbfdcbc

New changeset 2b9db1fce82e by R David Murray in branch '2.7':
#20135: FAQ entry for list mutation.
https://hg.python.org/cpython/rev/2b9db1fce82e
msg227799 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-09-29 14:26
I accidentally committed the patch early to 3.4/3.5.  I've now addressed Ezio's review comment per my suggestion on the review, and committed it to 2.7 as well.

Thanks everyone for your contributions.
History
Date User Action Args
2014-09-29 14:26:26r.david.murraysetstatus: open -> closed
resolution: fixed
messages: + msg227799

stage: commit review -> resolved
2014-09-29 14:24:34python-devsetmessages: + msg227798
2014-09-27 22:31:14r.david.murraysetfiles: + mutable_faq.patch

messages: + msg227741
2014-08-03 16:31:43francismbsetnosy: + francismb
messages: + msg224645
2014-08-03 13:26:20ezio.melottisetfiles: + issue20135-2.diff

messages: + msg224635
2014-07-11 23:31:55ezio.melottisetmessages: + msg222812
2014-07-11 19:48:44r.david.murraysetmessages: + msg222784
2014-07-07 09:57:13ezio.melottisetassignee: docs@python -> ezio.melotti

nosy: + rhettinger
stage: patch review -> commit review
2014-07-07 07:11:08berker.peksagsetnosy: + berker.peksag

messages: + msg222440
stage: needs patch -> patch review
2014-07-06 19:34:41BreamoreBoysetnosy: + BreamoreBoy

messages: + msg222413
versions: + Python 3.5, - Python 3.3
2014-07-06 19:15:04ezio.melottisetfiles: + issue20135.diff

messages: + msg222412
2014-07-06 17:58:07python-devsetnosy: + python-dev
messages: + msg222410
2014-06-03 02:31:16r.david.murraylinkissue21576 superseder
2014-03-24 14:41:15mvolzsetfiles: + sortFAQ_defaultval.patch

messages: + msg214688
2014-03-12 19:02:26r.david.murraysetmessages: + msg213277
2014-03-12 06:31:10eric.araujosetmessages: + msg213204
2014-03-10 19:18:11r.david.murraysetmessages: + msg213073
2014-03-10 18:21:49eric.araujosetnosy: + eric.araujo
messages: + msg213057
2014-03-10 17:47:24mvolzsetfiles: + defaultdoc_listmut.patch

messages: + msg213052
2014-03-09 23:34:09mvolzsetnosy: + mvolz
2014-01-17 15:36:16r.david.murraysetmessages: + msg208342
2014-01-17 14:27:45ezio.melottisetnosy: + ezio.melotti
messages: + msg208337
2014-01-17 05:26:13Fran.Bullsetfiles: + 20135.patch
keywords: + patch
2014-01-16 21:09:24Fran.Bullsetmessages: + msg208312
2014-01-16 20:25:43r.david.murraysetmessages: + msg208305
2014-01-16 18:46:55Fran.Bullsetnosy: + Fran.Bull
messages: + msg208297
2014-01-10 22:46:19terry.reedysettitle: mutate list -> FAQ need list mutation answers
stage: needs patch
2014-01-06 03:53:54r.david.murraysetversions: + Python 2.7, Python 3.4
nosy: + r.david.murray, docs@python

messages: + msg207411

assignee: docs@python
components: + Documentation
2014-01-06 03:37:24m123orningcreate