classification
Title: Add a "swap" method to list
Type: enhancement Stage:
Components: Interpreter Core Versions: Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: Rhamphoryncus, kristjan.jonsson, pitrou, r.david.murray, rhettinger
Priority: low Keywords: patch

Created on 2009-06-23 13:12 by kristjan.jonsson, last changed 2009-07-01 15:08 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
listswap.patch kristjan.jonsson, 2009-06-23 13:12
Messages (9)
msg89626 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-06-23 13:12
It is sometimes useful to be able to swap the contents of two lists and 
this patch provides a way to do so using the fastest way possible.

> a = [1, 2]
> b = [3]
> id(a), id(b)
(100, 101)
> a.swap(b)
> a
[3]
> b
[1, 2]
> id(a), id(b)
(100, 101)

One application of this is to make help a performance problem when one 
wants to upgrade a list instance into a subclass instance.
orglist = rawlist_from_server()
mylist = ListSubclass(orglist)

This involves copying duplicating the list, and then discarding, which 
can take a while for long lists.  Much faster is using>
mylist = ListSubclass()
mulist.swap(orglist)

We are using this extension within CCP to decoratate database queries 
from our database engine, that are returned as python lists.  The 
performance gained by shaving off this extra list duplication can be 
significant for large data sets.
to change a list, into a
msg89633 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-06-23 16:57
> One application of this is to make help a performance problem when one 
> wants to upgrade a list instance into a subclass instance.

Since this bypasses the subclass's __init__ and other methods, doesn't
it risk violating subclass invariants?

class CapList(list):
   def __init__(self, iterable=()):
       for elem in iterable:
           self.append(elem.upper())

class NoneCountingList(list):
   def __init__(self, iterable=()):
       list.__init__(self, iterable)
       self.nones = self.count(None)
   def append(self, value):
       list.append(self, value)
       self.nones += 1 if value is None else 0
   def extend(self, iterable):
       for elem in iterable:
           self.append(elem)
   . . .

IOW, a swap() method is problematic for some subclasses because it
bypasses all of the subclass insertion/removal logic.  The problem is
compounded for subclasses written as C extensions because violating the
internal invariants may lead to a crash.
msg89635 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-06-23 17:52
There are clearly enough subtleties to this proposal that it should be
discussed on python-ideas first.  If a consensus is reached you can
reopen this ticket, referencing the discussion thread.
msg89636 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-06-23 18:10
FWIW, the technique is useful and fast, but it is complicated in its
effects.  I used something similar in the set_swap_bodies() internal
code for set and frozenset objects but avoided exposing the behavior
externally.
msg89639 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-06-23 18:45
Indeed, I realized that it does allow overriding all kinds of behaviour 
and as such may be dangerous for the unwary.  But isn't it possible to 
do so anyway?

One way to increase safety would be to require that the "other" list is 
not a subclass (PyList_CheckExact(v)) so that no unexpected behaviour 
occurs for it, and so allow the 'target' list to simply override "swap" 
if it deems it unacceptable.

At any rate, I thought I'd share this idea and accept that it needs 
discussion. I'll start a thread on python-ideas.
msg89658 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-06-24 09:37
-1 from me. It is too marginal and obscure an use case to warrant a
dedicated method on a builtin type.
msg89957 - (view) Author: Adam Olsen (Rhamphoryncus) Date: 2009-06-30 23:52
Fix it at its source: patch your database engine to use the type you
want.  Or wrap the list without subclassing (__iter__ may be the only
method you need to wrap).

Obscure performance hacks don't warrant language extensions.
msg89958 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-07-01 00:03
fyi, here is the thread from python-ideas:

http://mail.python.org/pipermail/python-ideas/2009-June/005042.html

The parallel to C++´s future "rvalue reference" is interesting 
(http://www.artima.com/cppsource/rvalue.html, see the "move" 
application) 

Especially, perhaps, in light of this quote from http://evanjones.ca/rvalue-references.html, criticising the concept:

"...Or add a swap member function. This is not "elegant" nor is it as 
general. However, this is what we've been using for decades. It also 
requires zero changes to C++ compilers, and requires users to learn 
nothing. It has been immortalized in books like Effective C++ (Item 25: 
Consider support for a non-throwing swap)."

This, incidentally, is where the name 'swap' comes from, me being an old 
C++ programmer.
msg89982 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-07-01 15:08
Even if C++ decides it's an useful feature, it's not a sufficient reason
to add it to Python!
Based on previous discussion and the converging advice of others, this
bug can IMO be closed.
History
Date User Action Args
2009-07-01 15:08:10pitrousetstatus: open -> closed
keywords: patch, patch
resolution: rejected
messages: + msg89982
2009-07-01 00:03:23kristjan.jonssonsetkeywords: patch, patch

messages: + msg89958
2009-06-30 23:52:50Rhamphoryncussetnosy: + Rhamphoryncus
messages: + msg89957
2009-06-24 09:37:50pitrousetkeywords: patch, patch
nosy: + pitrou
messages: + msg89658

2009-06-23 19:34:11rhettingersetfiles: - mview2.diff
2009-06-23 19:33:19rhettingersetkeywords: patch, patch
files: + mview2.diff
2009-06-23 18:45:51kristjan.jonssonsetkeywords: patch, patch

messages: + msg89639
2009-06-23 18:10:42rhettingersetkeywords: patch, patch
status: pending -> open
messages: + msg89636
2009-06-23 17:52:13r.david.murraysetstatus: open -> pending
priority: low

nosy: + r.david.murray
messages: + msg89635

keywords: patch, patch
2009-06-23 16:57:08rhettingersetkeywords: patch, patch
nosy: + rhettinger
messages: + msg89633

2009-06-23 13:12:47kristjan.jonssoncreate