classification
Title: Make test_mailbox deterministic
Type: resource usage Stage: resolved
Components: Tests Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: neologix, pitrou, python-dev, r.david.murray, sdaoden
Priority: normal Keywords: buildbot, patch

Created on 2011-04-18 13:19 by r.david.murray, last changed 2011-12-20 10:50 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
mailbox_fork_with_ipc.patch r.david.murray, 2011-04-18 13:19 review
test_mailbox_evt.diff neologix, 2011-12-19 10:03 review
Messages (13)
msg133967 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-04-18 13:19
Attached is a patch to remove the last sleeps from test_mailbox.  I believe this makes the test suite deterministic.  It also shaves 4 seconds of fixed overhead off the test run time.
msg134053 - (view) Author: Steffen Daode Nurpmeso (sdaoden) Date: 2011-04-19 13:27
Nice ping pong you play..
I, buildbot get:

Using random seed 2215045
[1/1] test_mailbox
test test_mailbox failed -- multiple errors occurred; run in verbose mode for details
test test_mailbox failed -- Traceback (most recent call last):
  File "/Users/steffen/usr/opt/py3k/lib/python3.3/test/test_mailbox.py", line 1033, in test_lock_conflict
    ipc.signal('done')
  File "/Users/steffen/usr/opt/py3k/lib/python3.3/test/test_mailbox.py", line 1007, in __exit__
    self.c_sock.shutdown(socket.SHUT_RDWR)
socket.error: [Errno 57] Socket is not connected

test test_mailbox failed -- Traceback (most recent call last):
  File "/Users/steffen/usr/opt/py3k/lib/python3.3/test/test_mailbox.py", line 1033, in test_lock_conflict
    ipc.signal('done')
  File "/Users/steffen/usr/opt/py3k/lib/python3.3/test/test_mailbox.py", line 1007, in __exit__
    self.c_sock.shutdown(socket.SHUT_RDWR)
socket.error: [Errno 57] Socket is not connected

/Users/steffen/usr/opt/py3k/lib/python3.3/test/regrtest.py:1082: ResourceWarning: unclosed <socket.socket object, fd=5, family=2, type=1, proto=0>
  gc.collect()
1 test failed:
    test_mailbox
Re-running failed tests in verbose mode
Re-running test 'test_mailbox' in verbose mode

...

======================================================================
ERROR: test_lock_conflict (test.test_mailbox.TestMbox)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/steffen/usr/opt/py3k/lib/python3.3/test/test_mailbox.py", line 1033, in test_lock_conflict
    ipc.signal('done')
  File "/Users/steffen/usr/opt/py3k/lib/python3.3/test/test_mailbox.py", line 1007, in __exit__
    self.c_sock.shutdown(socket.SHUT_RDWR)
socket.error: [Errno 57] Socket is not connected

======================================================================
ERROR: test_lock_conflict (test.test_mailbox.TestMMDF)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/steffen/usr/opt/py3k/lib/python3.3/test/test_mailbox.py", line 1033, in test_lock_conflict
    ipc.signal('done')
  File "/Users/steffen/usr/opt/py3k/lib/python3.3/test/test_mailbox.py", line 1007, in __exit__
    self.c_sock.shutdown(socket.SHUT_RDWR)
socket.error: [Errno 57] Socket is not connected

----------------------------------------------------------------------
Ran 332 tests in 55.487s

FAILED (errors=2)
test test_mailbox failed -- multiple errors occurred
ok

... repeats yet another two times (very long output though)

----------------------------------------------------------------------
Ran 332 tests in 151.484s

OK
[118424 refs]
msg134056 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-04-19 14:11
Thanks for testing this.  I was afraid something like that would happen, since socket implementations are different on different platforms.  I presume you ran it on OSX.

Now I have to decide if I want to fix it, or if I should just switch to using threads.
msg134061 - (view) Author: Steffen Daode Nurpmeso (sdaoden) Date: 2011-04-19 14:35
Short glance into it, and after
commenting out 
    self.c_sock_close = self.c_sock_shutdown = True
in the parent process it ends up as
(maybe i can spend more time this evening):


15:36 ~/tmp $ python3 -E -Wd -m test -r -w -uall test_mailbox
Using random seed 9754656
[1/1] test_mailbox
/Users/steffen/usr/opt/py3k/lib/python3.3/mailbox.py:723:  ResourceWarning: unclosed <socket.socket object, fd=5, family=2, type=1, proto=0>
  return self._toc[key]
/Users/steffen/usr/opt/py3k/lib/python3.3/mailbox.py:77: ResourceWarning: unclosed <socket.socket object, fd=5, family=2, type=1, proto=0>
  return self.get_message(key)
1 test OK.
[118589 refs]
msg134072 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-04-19 15:45
I think the fix is to either put a try/except around the socket shutdown call, or to remove it entirely (I think things will still work right on linux without it).  If you leave the self.c_sock_close = True in, it should take care of the resource warning.
msg134138 - (view) Author: Steffen Daode Nurpmeso (sdaoden) Date: 2011-04-20 11:04
'Was not allowed to look yesterday.
If the child only closes and not
                self.c_sock_shutdown = True
(shutdown is an ugly word for a child anyway) then
(and i hope Apple did not modify the BSD network stack):

12:59 ~/tmp $ python3 -E -Wd -m test -r -w  test_mailbox
Using random seed 1985762
[1/1] test_mailbox
1 test OK.
msg149745 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-12-18 12:01
There was a recent buildbot failure on test_lock_conflict() because of a race.
Looking at your patch, I must be missing something, but why not simply use a multiprocessing condition to signal when the parent process has acquired the lock?
Otherwise you could simply use a pipe or a socketpair, it would be much simpler.
Also, there's a race: if the parent process calls connect() before the child calls listen(), he'll get ECONNREFUSED.
msg149814 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-12-19 04:13
Probably because I'm a threading/multiprocessing neophyte :)  (Though I just learned a bunch about programming with threads recently...)
msg149825 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-12-19 10:03
> Probably because I'm a threading/multiprocessing neophyte :)

That's a very good reason :-)

Here's a version using two multiprocessing events. Note that I use
timeouts for wait() just to avoid being stuck if something goes wrong:
the test now runs in a dozen ms.

By the way, next time you need a duplex communication between two
processes, you can use socketpair(), which returns a pair of connected
sockets.
msg149827 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-19 10:26
Charles-François's patch looks good to me.
msg149843 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-12-19 11:20
New changeset c6d41dd60d2d by Charles-François Natali in branch '2.7':
Issue #11867: Make test_mailbox.test_lock_conflict deterministic (and fix a
http://hg.python.org/cpython/rev/c6d41dd60d2d

New changeset 0053b7c68a02 by Charles-François Natali in branch '3.2':
Issue #11867: Make test_mailbox.test_lock_conflict deterministic (and fix a
http://hg.python.org/cpython/rev/0053b7c68a02

New changeset 020260ec44f2 by Charles-François Natali in branch 'default':
Issue #11867: Make test_mailbox.test_lock_conflict deterministic (and fix a
http://hg.python.org/cpython/rev/020260ec44f2
msg149851 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-12-19 12:34
Should be fixed now, thanks!
msg149910 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-12-20 10:50
New changeset c9facd251725 by Charles-François Natali in branch '2.7':
Followup to issue #11867: Use socketpair(), since FreeBSD < 8 doesn't really
http://hg.python.org/cpython/rev/c9facd251725

New changeset 9dee8a095faf by Charles-François Natali in branch '3.2':
Followup to issue #11867: Use socketpair(), since FreeBSD < 8 doesn't really
http://hg.python.org/cpython/rev/9dee8a095faf

New changeset 9014e0cc5589 by Charles-François Natali in branch 'default':
Followup to issue #11867: Use socketpair(), since FreeBSD < 8 doesn't really
http://hg.python.org/cpython/rev/9014e0cc5589
History
Date User Action Args
2011-12-20 10:50:02python-devsetmessages: + msg149910
2011-12-19 12:34:52neologixsetstatus: open -> closed
resolution: fixed
messages: + msg149851

stage: patch review -> resolved
2011-12-19 11:20:24python-devsetnosy: + python-dev
messages: + msg149843
2011-12-19 10:26:21pitrousetnosy: + pitrou
messages: + msg149827
2011-12-19 10:03:15neologixsetfiles: + test_mailbox_evt.diff

messages: + msg149825
2011-12-19 04:13:35r.david.murraysetmessages: + msg149814
2011-12-18 12:01:15neologixsetnosy: + neologix
messages: + msg149745
2011-04-20 11:04:13sdaodensetmessages: + msg134138
2011-04-19 15:45:37r.david.murraysetmessages: + msg134072
2011-04-19 14:35:42sdaodensetmessages: + msg134061
2011-04-19 14:11:40r.david.murraysetmessages: + msg134056
2011-04-19 13:27:49sdaodensetnosy: + sdaoden
messages: + msg134053
2011-04-18 13:31:52r.david.murraysetversions: + Python 2.7
2011-04-18 13:19:04r.david.murraycreate