classification
Title: test_ssl freezes
Type: Stage:
Components: Library (Lib), Tests Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, pitrou, richard
Priority: normal Keywords:

Created on 2010-08-15 23:20 by pitrou, last changed 2010-08-16 01:48 by richard. This issue is now closed.

Messages (5)
msg114024 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-15 23:20
Recently there have been test_ssl freezes on the buildbots. They seem to happen in the asyncore test case:

http://www.python.org/dev/buildbot/builders/i386%20Ubuntu%203.x/builds/1903/steps/test/logs/stdio
http://www.python.org/dev/buildbot/builders/x86%20Ubuntu%203.x/builds/1742/steps/test/logs/stdio

test_asyncore_server (test.test_ssl.ThreadedTests)
Check the example asyncore integration. ... 
 server:  new connection from 127.0.0.1:36622
 client:  sending b'FOO\n'...
 server:  read b'FOO\n' from client
 client:  read b'foo\n'
 client:  closing connection.
 server:  read b'over\n' from client
 server:  closed connection <ssl.SSLSocket [closed]  object, fd=-1, family=2, type=1, proto=0>
 server:  read b'' from client


The only significant change recently in ssl has been r83869, and asyncore doesn't seem to have recent any important changes lately.
msg114029 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-16 00:17
Actually, this can be reproduced by running test_smtpd before test_ssl:

$ ./python -m test.regrtest -v -uall test_smtpd test_ssl

It then freezes at that point:

[...]
test_asyncore_server (test.test_ssl.ThreadedTests)
Check the example asyncore integration. ... 
 server:  new connection from 127.0.0.1:46707
 client:  sending b'FOO\n'...
 server:  read b'FOO\n' from client
 client:  read b'foo\n'
 client:  closing connection.
 server:  read b'over\n' from client
 client:  connection closed.
 cleanup: stopping server.
 cleanup: joining server thread.
 server:  closed connection <ssl.SSLSocket [closed]  object, fd=-1, family=2, type=1, proto=0>
 server:  read b'' from client

If you press the return key, it unfreezes the test (does test_smtpd register file descriptor 0 (stdin) in asyncore?) and prints the following error:

error: uncaptured python exception, closing channel <smtpd.SMTPChannel connected 'c' at 0x2c4c370> (<class 'IndexError'>:pop from empty list [/home/antoine/py3k/debug/Lib/asyncore.py|read|79] [/home/antoine/py3k/debug/Lib/asyncore.py|handle_read_event|435] [/home/antoine/py3k/debug/Lib/asynchat.py|handle_read|128] [/home/antoine/py3k/debug/Lib/asyncore.py|recv|375] [/home/antoine/py3k/debug/Lib/test/mock_socket.py|recv|47])
msg114030 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-16 00:19
> does test_smtpd register file descriptor 0 (stdin) in asyncore?

After some debug prints, it turns out to be the case.
One general problem is asyncore's global socket_map, which means it can leak between tests if some tests fail cleaning correctly after themselves.
msg114031 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-16 00:29
In r84086, I've added a save/restore guard of asyncore.socket_map to the regression test suite. It also warns when a test fails to leave the socket_map in its initial test. test_smtpd is the only test that displays the warning.
msg114033 - (view) Author: Richard Jones (richard) * (Python committer) Date: 2010-08-16 01:48
Thanks for the investigation Antoine.

In r84088 I've added a call to asyncore.close_all in the smtpd test tearDown methods.
History
Date User Action Args
2010-08-16 01:48:46richardsetstatus: open -> closed
resolution: fixed
messages: + msg114033
2010-08-16 00:29:55pitrousetmessages: + msg114031
2010-08-16 00:19:41pitrousetmessages: + msg114030
2010-08-16 00:17:32pitrousetnosy: + richard
messages: + msg114029
2010-08-15 23:20:50pitroucreate