classification
Title: mailbox.mbox fails to pop two items in a row, flushing in between
Type: behavior Stage: resolved
Components: email, Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: barry, petri.lehtinen, python-dev, r.david.murray
Priority: normal Keywords: patch

Created on 2012-06-08 14:18 by petri.lehtinen, last changed 2012-06-18 07:50 by petri.lehtinen. This issue is now closed.

Files
File name Uploaded Description Edit
issue15036.patch petri.lehtinen, 2012-06-15 12:12
Messages (10)
msg162528 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2012-06-08 14:18
test_mbox is an mbox mailbox with a few messages in it.

>>> import mailbox
>>> inbox = mailbox.mbox('test_mbox')
>>> inbox.lock()
>>> inbox.popitem()
(0, <mailbox.mboxMessage instance at 0x7f78016bc680>)
>>> inbox.flush()
>>> inbox.unlock()
>>> inbox.lock()
>>> inbox.popitem()
(1, <mailbox.mboxMessage instance at 0x7f7801653320>)
>>> inbox.flush()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/mailbox.py", line 633, in flush
    (self._file_length, cur_len))
mailbox.ExternalClashError: Size of mailbox file changed (expected 141289, found 141147)
msg162529 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2012-06-08 14:20
Actually, you don't even need to unlock() and lock() the mailbox  before the second popitem() and flush().
msg162811 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2012-06-14 18:19
The fix seems to be very simple:

diff --git a/Lib/mailbox.py b/Lib/mailbox.py
index a677729..2be4c83 100644
--- a/Lib/mailbox.py
+++ b/Lib/mailbox.py
@@ -675,6 +675,7 @@ class _singlefileMailbox(Mailbox):
                     new_file.write(buffer)
                 new_toc[key] = (new_start, new_file.tell())
                 self._post_message_hook(new_file)
+            self._file_length = new_file.tell()
         except:
             new_file.close()
             os.remove(new_file.name)

I guess all single-file mailboxes have this issue, not only mbox. I'll still need to add tests when I have time.
msg162890 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2012-06-15 12:12
As I suspected, all single-file mailboxes(mbox, MMDF, Babyl) have this issue. Attached a patch with tests.
msg162926 - (view) Author: Roundup Robot (python-dev) Date: 2012-06-15 18:04
New changeset 0add70dd3c43 by Petri Lehtinen in branch '2.7':
#15036: Make a repeated changes and flushes work with single-file mailboxes
http://hg.python.org/cpython/rev/0add70dd3c43

New changeset 714b8f91f3d4 by Petri Lehtinen in branch '3.2':
#15036: Make a repeated changes and flushes work with single-file mailboxes
http://hg.python.org/cpython/rev/714b8f91f3d4

New changeset 87d119117560 by Petri Lehtinen in branch 'default':
#15036: Make a repeated changes and flushes work with single-file mailboxes
http://hg.python.org/cpython/rev/87d119117560
msg162928 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-06-15 18:38
The news item isn't completely clear.  It sounds like the mailbox is now automatically being flushed between pops, but what you really fixed is popping if the *application* does a flush between them, right?
msg162982 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2012-06-16 18:18
Yes, this is what I tried to say. It's hard for me to find a good
wording, so what would you suggest?

I also noticed now that there's a typo in the commit messages. But
those cannot be fixed anymore.
msg162986 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-06-16 18:45
"Mailbox no longer throws an error if a flush is done between operations when removing or changing multiple items in mbox, MMDF, or Babyl mailboxes."
msg163090 - (view) Author: Roundup Robot (python-dev) Date: 2012-06-18 07:49
New changeset 8b38a81ba3bf by Petri Lehtinen in branch '2.7':
Fix NEWS entry for #15036
http://hg.python.org/cpython/rev/8b38a81ba3bf

New changeset 38e2a87c9051 by Petri Lehtinen in branch '3.2':
Fix NEWS entry for #15036
http://hg.python.org/cpython/rev/38e2a87c9051

New changeset 072b08989731 by Petri Lehtinen in branch 'default':
Fix NEWS entry for #15036
http://hg.python.org/cpython/rev/072b08989731
msg163091 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2012-06-18 07:50
Perfect, fixed.
History
Date User Action Args
2012-06-18 07:50:25petri.lehtinensetmessages: + msg163091
2012-06-18 07:49:30python-devsetmessages: + msg163090
2012-06-16 18:45:43r.david.murraysetmessages: + msg162986
2012-06-16 18:18:48petri.lehtinensetmessages: + msg162982
2012-06-15 18:38:54r.david.murraysetmessages: + msg162928
2012-06-15 18:05:30petri.lehtinensetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2012-06-15 18:04:01python-devsetnosy: + python-dev
messages: + msg162926
2012-06-15 12:12:03petri.lehtinensetfiles: + issue15036.patch
keywords: + patch
messages: + msg162890
2012-06-14 18:19:04petri.lehtinensetmessages: + msg162811
2012-06-08 18:12:31r.david.murraysetnosy: + barry, r.david.murray

type: behavior
components: + email
stage: needs patch
2012-06-08 14:20:07petri.lehtinensetmessages: + msg162529
2012-06-08 14:18:55petri.lehtinencreate