classification
Title: UserList-subclass Tree slicing changes the original list unexpectedly
Type: behavior Stage: resolved
Components: Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: ctarn, eric.snow, mark.dickinson, steven.daprano, terry.reedy
Priority: normal Keywords:

Created on 2019-12-20 15:00 by ctarn, last changed 2019-12-21 12:43 by ctarn. This issue is now closed.

Files
File name Uploaded Description Edit
bug.py ctarn, 2019-12-21 07:20
Messages (14)
msg358709 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-20 15:06
Sorry, it is caused by list(). I will update the issue very soon.
msg358710 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-20 15:23
In the code, each item of ls = [[0], [1], [2],...] has an owner pointing to d, which is a Tree inheriting from collections.UserList.
When `d[0] = a`, and `a.owner = d`, and `_ = list(d[0:1])` is called, a.owner will be changed to d[0:1].
msg358711 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2019-12-20 15:24
Can you give an example? Something simple, showing what you tried, what you expected, and what happened instead.

This may help: http://sscce.org/

Your bug.py file doesn't ever use the result of calling list(), so how do you know it changes the value of the parameter?
msg358712 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-20 15:26
Hi, thanks. It did call `list()` through `_ = list(d[0:2])`
msg358713 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-20 15:30
I printed the value of *.owner before and after `_ = list(d...)`, and marked the results in the comments.
msg358714 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-20 15:38
OK, I mean... when I call `a = list(b)`, list() changes `b` unexpectedly, not `a != b` in this case. That is why I replace the left operand with `_`.
msg358725 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-12-20 18:12
Your problem is with UserList.  This is from the implementation:

    def __getitem__(self, i):
        if isinstance(i, slice):
            return self.__class__(self.data[i])
        else:
            return self.data[i]

So each slice is creating a new Tree.  Then the __setitem__() of that new Tree gets triggered for each item, causing the owner to be set to the new Tree.  So that's the cause.

Is there a particular reason you are subclassing UserList?  Consider subclassing list or collections.abc.MutableSequence instead.  Or deal with the slice semantics manually.

Also, perhaps you expected that a slice would produce a view into the sequence?  I know that's how it works in some programming languages (but not Python, though you could make it do that if you wanted to).

Regardless, as far as I can tell everything is working as designed.  I recommend closing this.
msg358741 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-12-20 21:59
(ctarn), if you want to discuss what you are doing further, please try a discussion list, such as python-list.
msg358756 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-21 04:54
Sorry but it is not. See the first time I print ls[4].owner. We get d as expected, not a slice of d, that is, d[0:2].

However the next time we print it after _ = list(d[0:1]), noticed that ls[4] == d[0:1], we get d[0:1], it’s extremely surprising!!!

I have to highlight it: we just print *.owner before and after `_ = list(d[0:1])`, and the results are different!!!

The latest 3 lines show more strange results.

By the way, it’s my first time to report bug, and I don’t know what is discussion list, and reopened the issue. Thank you.
msg358763 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-21 07:20
I tried to remove list(), and just use cmd like `_ = d[0:1]`, and it surprisingly changed d[0].owner from d to d[0:1]!
The file attached is updated.
I pretty sure that it is a (serious) bug. Please check it more carefully. Thanks.
msg358764 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-21 07:25
Moreover, it works as expected with Python 3.6 (the owner of each of them is d), and Python 3.8 and Python 3.7 work differently.
I didn't try it using Python 3.9 yet.
msg358765 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-21 07:56
and more... It doesn't happen when Tree directly subclasses list.
msg358768 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2019-12-21 11:55
[ctarn]

> it works as expected with Python 3.6 (the owner of each of them is d), and Python 3.8 and Python 3.7 work differently

The change in behaviour is the result of a bug fix that was applied in 3.7 and upwards: see GH-13169 and #27639.

As Eric says, UserList is behaving as intended here. Your problem stems from a design flaw in your code, namely that Nodes have owners. If a node exists both in a Tree *and* in a slice of that Tree (which slice, since the #27639 fix, is again a Tree), that node can't have both the original Tree *and* the slice as owner. In this case, what's happening is that accessing `d[0:1]` creates a new Tree object, and the __init__ method for that Tree object then reassigns the "owner" of the node in that slice.

> I don’t know what is discussion list

See https://www.python.org/community/lists/, and particularly https://mail.python.org/mailman/listinfo/python-list

Closing again here, but feel free to start a discussion or ask questions on the list above.
msg358769 - (view) Author: Tarn Yeong Ching (ctarn) Date: 2019-12-21 12:43
GOD... I see. Thank you very much!
History
Date User Action Args
2019-12-21 12:43:31ctarnsetmessages: + msg358769
2019-12-21 11:55:33mark.dickinsonsetstatus: open -> closed

nosy: + mark.dickinson
messages: + msg358768

resolution: not a bug
2019-12-21 07:56:29ctarnsetmessages: + msg358765
2019-12-21 07:25:51ctarnsetmessages: + msg358764
versions: + Python 3.7, - Python 3.9
2019-12-21 07:24:21ctarnsetfiles: - bug.py
2019-12-21 07:20:51ctarnsetfiles: + bug.py

messages: + msg358763
title: UserList-subclass Tree slicing changes node attribute -> UserList-subclass Tree slicing changes the original list unexpectedly
2019-12-21 04:55:25ctarnsetresolution: not a bug -> (no value)
2019-12-21 04:54:51ctarnsetstatus: closed -> open

messages: + msg358756
2019-12-20 21:59:10terry.reedysetnosy: + terry.reedy
messages: + msg358741
2019-12-20 21:57:45terry.reedysetstatus: pending -> closed
stage: resolved
2019-12-20 21:56:49terry.reedysetstatus: open -> pending
2019-12-20 21:56:23terry.reedysetstatus: pending -> open
title: It seems that list() changes the value of the parameter -> UserList-subclass Tree slicing changes node attribute
2019-12-20 18:12:26eric.snowsetstatus: open -> pending
versions: + Python 3.9
nosy: + eric.snow

messages: + msg358725

resolution: not a bug
2019-12-20 15:38:19ctarnsetmessages: + msg358714
2019-12-20 15:30:36ctarnsetmessages: + msg358713
2019-12-20 15:26:53ctarnsetmessages: + msg358712
2019-12-20 15:24:44steven.dapranosetnosy: + steven.daprano
messages: + msg358711
2019-12-20 15:23:33ctarnsetmessages: + msg358710
2019-12-20 15:17:39ctarnsetfiles: + bug.py
title: It seems that unittest.TestCase.assertListEqual changes the value of its parameters -> It seems that list() changes the value of the parameter
2019-12-20 15:15:30ctarnsetfiles: - bug.py
2019-12-20 15:06:58ctarnsetnosy: + ctarn
messages: + msg358709
2019-12-20 15:04:44ctarnsettitle: It seems that TestCase.assertListEqual change the value of its parameters -> It seems that unittest.TestCase.assertListEqual changes the value of its parameters
2019-12-20 15:02:33ctarnsetnosy: - ctarn
versions: + Python 3.8
-> (no value)
title: It -> It seems that TestCase.assertListEqual change the value of its parameters
2019-12-20 15:00:48ctarncreate