classification
Title: mailbox.Mailbox does odd hasattr() check
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Rosuav, barry, peter.otten, petri.lehtinen, python-dev, r.david.murray, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2014-02-22 12:45 by Rosuav, last changed 2014-08-13 07:00 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
mailbox_iters.patch serhiy.storchaka, 2014-02-22 19:05 review
Messages (8)
msg211920 - (view) Author: Chris Angelico (Rosuav) * Date: 2014-02-22 12:45
Only noticed because I was searching the stdlib for hasattr calls, but in mailbox.Mailbox.update(), a check is done thus:

        if hasattr(arg, 'iteritems'):
            source = arg.items()
        elif hasattr(arg, 'items'):
            source = arg.items()
        else:
            source = arg

If this is meant to support Python 2, it should probably use iteritems() in the first branch, but for Python 3, it's probably simpler to just drop the first check altogether:

        if hasattr(arg, 'items'):
            source = arg.items()
        else:
            source = arg

Or possibly switch to EAFP:

        try:
            source = arg.items()
        except AttributeError:
            source = arg
msg211939 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-22 19:05
Actually it should be iteritems(). This is meant to support Mailbox, which has both iteritems() and items() methods. iteritems() returns an iterator and items() returns a list. Looks as changeset f340cb045bf9 was incorrectly applied to mailbox. Here is a patch which partially reverts changeset f340cb045bf9 for the mailbox module and fixes tests in 3.x.

Perhaps we should change items() to return an iterator in Python 4.0.
msg211951 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-02-22 21:01
You have opened the door on a slight mess. The mailbox module provides a set + dict interface to on-disk mailbax files in various formats. The hg annotate and revision history commands indicate that most of 3.4 mailbox is unchanged since the trunk version was merged into py3k on 2006-5-27 in rev38453. However, on 2007-2-11 Guido changed .iterxxx to .xxx throughout the stdlib in rev40809.

The bug you note is a side-effect of this patch. It overall left mailbax in a somewhat inconsistent state as it did *not* update the mailbox dict API by removing the mailbox.iterkeys/values/items methods and replacing the .keys/values/items methods. As a result, the mailbox methods that space efficiently iterated thru self.iterkeys now iterate through self.keys == list(self.iterkeys). Example:
    def clear(self):
        """Delete all messages."""
        for key in self.keys():  # was self.iterkeys()
            self.discard(key)

To fix this, I think we should either
1) revert the rev40809 changes to mailbox, including in the line you point to, or
2) complete the rev40809 changes by updating to a Py3 interface, and make the change you suggest.

1) is much easier, but the API looks odd to a py3-only programmer.
After writing this message, I see that Serhiy wrote a patch to do this.

2) is an api change that perhaps should have happened in 3.0. Deprecation is awkward since people should not change from, for instance, self.iterkeys to self.key, until the api change in made.
msg211959 - (view) Author: Peter Otten (peter.otten) * Date: 2014-02-22 21:19
Do you expect many use cases that rely on items(), keys(), and values() being lists? 
Maybe it would be acceptable to make these lazy in 3.5, but keep the iterXXX() variants as aliases indefinitely.
msg211960 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-22 21:40
Note that there is a difference between Mailbox and dict interface: __iter__() 
iterates over values, not keys.

clear() should use keys(), not iterkeys(), because it modifies iterated dict.
msg225242 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-12 20:00
If there are no objections I'll commit this patch.
msg225244 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-08-12 21:06
Committing the patch seems like the right thing to do at this point in time.
msg225256 - (view) Author: Roundup Robot (python-dev) Date: 2014-08-13 06:37
New changeset 5fd1f8271e8a by Serhiy Storchaka in branch '3.4':
Issue #20729: Restored the use of lazy iterkeys()/itervalues()/iteritems()
http://hg.python.org/cpython/rev/5fd1f8271e8a

New changeset acb30ed7eceb by Serhiy Storchaka in branch 'default':
Issue #20729: Restored the use of lazy iterkeys()/itervalues()/iteritems()
http://hg.python.org/cpython/rev/acb30ed7eceb
History
Date User Action Args
2014-08-13 07:00:11serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-08-13 06:37:32python-devsetnosy: + python-dev
messages: + msg225256
2014-08-12 21:06:18r.david.murraysetmessages: + msg225244
2014-08-12 20:00:09serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg225242
versions: + Python 3.5, - Python 3.3
2014-02-22 21:40:37serhiy.storchakasetmessages: + msg211960
2014-02-22 21:19:32peter.ottensetnosy: + peter.otten
messages: + msg211959
2014-02-22 21:01:05terry.reedysetnosy: + terry.reedy
messages: + msg211951
2014-02-22 19:05:45serhiy.storchakasetfiles: + mailbox_iters.patch

type: behavior
components: + Library (Lib)
versions: + Python 3.3, Python 3.4
keywords: + patch
nosy: + barry, serhiy.storchaka, r.david.murray, petri.lehtinen

messages: + msg211939
stage: patch review
2014-02-22 12:45:39Rosuavcreate