classification
Title: calling MutableSequence.extend on self produces infinite loop
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Naris R, rhettinger, serhiy.storchaka, stutzbach
Priority: normal Keywords: patch

Created on 2018-08-18 09:58 by Naris R, last changed 2018-08-30 16:56 by rhettinger. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8813 merged Naris R, 2018-08-19 02:04
Messages (8)
msg323696 - (view) Author: Naris R (Naris R) * Date: 2018-08-18 09:58
Example:

```
from typing import MutableSequence, TypeVar

CliffordGate = TypeVar('CliffordGate')

class QCircuit(MutableSequence[CliffordGate]):
    def __init__(self, gates):
        self.gates = list(gates)

    def __repr__(self):
        return f'{self.__class__.__name__}({self.gates})'
   
    def __getitem__(self, key):
        return self.gates[key]

    def __setitem__(self, key, item):
        self.gates[key] = item

    def __delitem__(self, key):
        del self.gates[key]

    def insert(self, key, item):
        self.gates.insert(key, item)

a = QCircuit(['H0', 'S2'])
a += a
```
msg323697 - (view) Author: Naris R (Naris R) * Date: 2018-08-18 10:07
I forgot to copy over __len__ in the example
msg323711 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-08-18 13:15
It would be reasonable to add special handling for this case to match what is done in the concrete sequences like list, deque, bytes, ...
msg323712 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-08-18 13:48
deque and bytearray make an in-memory copy of self if extended with self. deque creates a temporary list, bytearray creates a temporary bytearray. list just iterates itself as a linear array of known size and avoids infinite loop. It could be possible to avoid creaing a temporary copy in case of deque and bytearray too, but I don't think this special case is worth an additional code.

But a MutableSequence can represents a sequence that doesn't fit in memory. It can provide an interface to a linear on-disk store. In this case creating an on-memory copy is not possible.
msg324147 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-08-27 06:39
> But a MutableSequence can represents a sequence that doesn't
> fit in memory. It can provide an interface to a linear 
> on-disk store. In this case creating an on-memory copy 
> is not possible.

This case is likely not worth worrying about.  If a user created such a sequence AND wrote "s += s", it is unclear whether any particular unpleasant outcome (including the current behavior) could be avoided.
msg324160 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-08-27 09:45
Agreed. Appending >2**31 items one by one to the collection with external storage is not very efficient, so such collection likely has specialized extend().
msg324393 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-08-30 16:56
New changeset 1b5f9c9653f348b0aa8b7ca39f8a9361150f7dfc by Raymond Hettinger (Naris R) in branch 'master':
bpo-34427: Fix infinite loop when calling MutableSequence.extend() on self (GH-8813)
https://github.com/python/cpython/commit/1b5f9c9653f348b0aa8b7ca39f8a9361150f7dfc
msg324394 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-08-30 16:56
Thank you for the patch.
History
Date User Action Args
2018-08-30 16:56:54rhettingersetstatus: open -> closed
resolution: fixed
messages: + msg324394

stage: patch review -> resolved
2018-08-30 16:56:20rhettingersetmessages: + msg324393
2018-08-27 09:45:57serhiy.storchakasetmessages: + msg324160
2018-08-27 06:39:18rhettingersetmessages: + msg324147
2018-08-25 01:51:57terry.reedysetversions: + Python 3.6, Python 3.8
2018-08-19 02:04:01Naris Rsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8291
2018-08-18 13:48:05serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg323712
2018-08-18 13:15:48rhettingersetassignee: rhettinger
messages: + msg323711
2018-08-18 10:08:26Naris Rsetnosy: + rhettinger, stutzbach
2018-08-18 10:07:27Naris Rsetmessages: + msg323697
2018-08-18 09:58:31Naris Rcreate