classification
Title: test_asyncore failing on Solaris 10 2.7 (nitrogen/s10)
Type: Stage:
Components: Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: trent Nosy List: giampaolo.rodola, jcea, python-dev, trent
Priority: normal Keywords: patch

Created on 2012-10-18 08:06 by trent, last changed 2012-10-27 02:24 by trent. This issue is now closed.

Files
File name Uploaded Description Edit
issue16274_asyncore.patch trent, 2012-10-18 10:02
Messages (16)
msg173237 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-10-18 08:06
% ./python -m test.regrtest test_asyncore
test_asyncore
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/home/cpython/hg/2.7/Lib/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/home/cpython/hg/2.7/Lib/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/cpython/hg/2.7/Lib/test/test_asyncore.py", line 712, in <lambda>
    t = threading.Thread(target=lambda: asyncore.loop(timeout=0.1, count=500))
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 220, in loop
    poll_fun(timeout, map)
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 156, in poll
    read(obj)
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 87, in read
    obj.handle_error()
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 83, in read
    obj.handle_read_event()
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 443, in handle_read_event
    self.handle_accept()
  File "/home/cpython/hg/2.7/Lib/test/test_asyncore.py", line 487, in handle_accept
    sock, addr = self.accept()
TypeError: 'NoneType' object is not iterable

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/home/cpython/hg/2.7/Lib/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/home/cpython/hg/2.7/Lib/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/cpython/hg/2.7/Lib/test/test_asyncore.py", line 712, in <lambda>
    t = threading.Thread(target=lambda: asyncore.loop(timeout=0.1, count=500))
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 220, in loop
    poll_fun(timeout, map)
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 156, in poll
    read(obj)
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 87, in read
    obj.handle_error()
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 83, in read
    obj.handle_read_event()
  File "/home/cpython/hg/2.7/Lib/asyncore.py", line 443, in handle_read_event
    self.handle_accept()
  File "/home/cpython/hg/2.7/Lib/test/test_asyncore.py", line 487, in handle_accept
    sock, addr = self.accept()
TypeError: 'NoneType' object is not iterable

1 test OK.
[43391 refs]

Sample buildbot run that failed: http://buildbot.python.org/all/builders/SPARC64%20Solaris%2010%20%5BSB%5D%202.7/builds/69/steps/test/logs/stdio

Haven't investigated yet.
msg173244 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-10-18 09:19
That is weird. accept() should never return None.
What about python 3.x?
msg173246 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-10-18 09:30
3.2 appears fine.  This behavior only seems to happen on 2.7.

The affected tests are:

test_quick_connect (test.test_asyncore.TestAPI_UseSelect) 
test_quick_connect (test.test_asyncore.TestAPI_UsePoll) 

Doing some more digging.
msg173249 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-10-18 09:56
I notice there are some differences in sock_accept() (Modules/socketmodule.c) between 2.7 and 3.2.
Maybe a bugfix which wasn't backported to 2.7.
msg173250 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-10-18 09:57
Yeah I've just backported the semantic changes between asyncore and test_asyncore from 3.2 -> 2.7.

I'll report back shortly with results.
msg173251 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-10-18 10:02
Ok, attached patch is a semantic backport matching 2.7 to 3.2.  All tests pass with this patch.

(I can't comment on whether or not the changes in 3.2 actually fix an underlying issue, or just hide it better.)
msg173252 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-10-18 10:08
New changeset 90a46f8943d0 by Trent Nelson in branch '2.7':
Issue #16274: backport of 3.2's asyncore/test_asyncore to 2.7.
http://hg.python.org/cpython/rev/90a46f8943d0
msg173253 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-10-18 10:09
Patch applied and everything passes, so closing for now.
msg173255 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-10-18 11:45
New changeset 2f0770cc6d3f by Trent Nelson in branch '2.7':
Issue #16274: revert r79813:90a46f8943d0 changes to asyncore/test_asyncore.
http://hg.python.org/cpython/rev/2f0770cc6d3f
msg173256 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-10-18 11:47
That backport wasn't appropriate -- it included new public members to asyncore that were introduced in 3.2.

I'll work on a less intrusive patch.
msg173350 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2012-10-19 18:20
"hg annotate" is useful :-)

Changes in 3.2 look like coming from here:

"""
jcea@ubuntu:~/hg/python/cpython$ hg log -r 64988
changeset:   64988:0f8c2eb89f46
user:        Antoine Pitrou <solipsis@pitrou.net>
date:        Tue Sep 28 21:23:11 2010 +0000
summary:     Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or EAGAIN,

jcea@ubuntu:~/hg/python/cpython$ hg log -r 65314
changeset:   65314:12442ac3f7dd
parent:      65312:a27fc0ff3678
user:        Antoine Pitrou <solipsis@pitrou.net>
date:        Thu Oct 14 15:05:38 2010 +0000
summary:     Issue #7523: Add SOCK_CLOEXEC and SOCK_NONBLOCK to the socket module,
"""
msg173510 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2012-10-22 12:46
Keeping the issue open until Trent can commit a new patch, according to msg173256.
msg173511 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-10-22 12:51
This should do it:

diff --git a/Lib/test/test_asyncore.py b/Lib/test/test_asyncore.py
--- a/Lib/test/test_asyncore.py
+++ b/Lib/test/test_asyncore.py
@@ -484,8 +484,9 @@
         return self.socket.getsockname()[:2]
 
     def handle_accept(self):
-        sock, addr = self.accept()
-        self.handler(sock)
+        pair = self.accept()
+        if pair is not None:
+            self.handler(pair[0])
 
     def handle_error(self):
         raise
msg173512 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-10-22 12:58
Yes, now that I think of it that is how accept() is supposed to be used in 2.7 (back then I did also update doc examples in order to enforce this use case) so this is a test suite issue.
msg173671 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-10-24 12:13
On Mon, Oct 22, 2012 at 05:51:23AM -0700, Giampaolo Rodola' wrote:
> 
> Giampaolo Rodola' added the comment:
> 
> This should do it:
> 
> diff --git a/Lib/test/test_asyncore.py b/Lib/test/test_asyncore.py
> --- a/Lib/test/test_asyncore.py
> +++ b/Lib/test/test_asyncore.py
> @@ -484,8 +484,9 @@
>          return self.socket.getsockname()[:2]
>  
>      def handle_accept(self):
> -        sock, addr = self.accept()
> -        self.handler(sock)
> +        pair = self.accept()
> +        if pair is not None:
> +            self.handler(pair[0])
>  
>      def handle_error(self):
>          raise

    Yeah this looks a lot more appropriate than the two changes
    Jesús posted.  I'll test against my Solaris boxes and report
    back.
msg173909 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-10-27 02:21
New changeset 937fa81500e2 by Trent Nelson in branch '2.7':
Issue #16274: Fix test_asyncore on Solaris.
http://hg.python.org/cpython/rev/937fa81500e2
History
Date User Action Args
2012-10-27 02:24:38trentsetstatus: open -> closed
resolution: fixed
2012-10-27 02:21:53python-devsetmessages: + msg173909
2012-10-24 12:13:16trentsetmessages: + msg173671
2012-10-22 12:58:08giampaolo.rodolasetmessages: + msg173512
2012-10-22 12:51:23giampaolo.rodolasetmessages: + msg173511
2012-10-22 12:46:50jceasetmessages: + msg173510
2012-10-19 18:25:16trentsetstatus: closed -> open
2012-10-19 18:20:19jceasetmessages: + msg173350
2012-10-18 11:47:28trentsetmessages: + msg173256
2012-10-18 11:45:20python-devsetmessages: + msg173255
2012-10-18 10:09:15trentsetstatus: open -> closed

messages: + msg173253
2012-10-18 10:08:13python-devsetnosy: + python-dev
messages: + msg173252
2012-10-18 10:02:21trentsetfiles: + issue16274_asyncore.patch
keywords: + patch
messages: + msg173251
2012-10-18 09:57:08trentsetmessages: + msg173250
2012-10-18 09:56:12giampaolo.rodolasetmessages: + msg173249
2012-10-18 09:30:12trentsetmessages: + msg173246
2012-10-18 09:19:28giampaolo.rodolasetnosy: + giampaolo.rodola
messages: + msg173244
2012-10-18 08:06:51trentcreate