classification
Title: asyncore handling of out-of-band data fails
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: neologix, xdegaye
Priority: normal Keywords: patch

Created on 2011-11-01 16:04 by xdegaye, last changed 2011-11-02 14:17 by neologix. This issue is now closed.

Files
File name Uploaded Description Edit
handle_expt.diff xdegaye, 2011-11-01 16:04 review
Messages (4)
msg146783 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2011-11-01 16:04
Add the following lines to test_handle_expt (this makes sense, a
dispatcher instance is supposed to implement handle_read and call recv
in order to detect that the remote end has closed the socket):

--- a/Lib/test/test_asyncore.py
+++ b/Lib/test/test_asyncore.py
@@ -677,6 +677,9 @@
             def handle_expt(self):
                 self.flag = True
 
+            def handle_read(self):
+                self.recv(1)
+
         class TestHandler(BaseTestHandler):
             def __init__(self, conn):
                 BaseTestHandler.__init__(self, conn)

With these lines added, the test now fails on linux with Python 3.3,
see the following backtrace: select (an poll) returns a read event and
an exceptional condition for the socket, but there is no normal data
to read, only out-of-band data.

The attached patch fixes the problem.


======================================================================
ERROR: test_handle_expt (test.test_asyncore.TestAPI_UseIPv4Poll)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/path_to/src/cpython/cpython-hg-default/Lib/test/test_asyncore.py", line 690, in test_handle_expt
    self.loop_waiting_for_flag(client)
  File "/path_to/src/cpython/cpython-hg-default/Lib/test/test_asyncore.py", line 523, in loop_waiting_for_flag
    asyncore.loop(timeout=0.01, count=1, use_poll=self.use_poll)
  File "/path_to/src/cpython/cpython-hg-default/Lib/asyncore.py", line 215, in loop
    poll_fun(timeout, map)
  File "/path_to/src/cpython/cpython-hg-default/Lib/asyncore.py", line 196, in poll2
    readwrite(obj, flags)
  File "/path_to/src/cpython/cpython-hg-default/Lib/asyncore.py", line 117, in readwrite
    obj.handle_error()
  File "/path_to/src/cpython/cpython-hg-default/Lib/asyncore.py", line 108, in readwrite
    obj.handle_read_event()
  File "/path_to/src/cpython/cpython-hg-default/Lib/asyncore.py", line 439, in handle_read_event
    self.handle_read()
  File "/path_to/src/cpython/cpython-hg-default/Lib/test/test_asyncore.py", line 681, in handle_read
    self.recv(1)
  File "/path_to/src/cpython/cpython-hg-default/Lib/asyncore.py", line 379, in recv
    data = self.socket.recv(buffer_size)
BlockingIOError: [Errno 11] Resource temporarily unavailable
msg146806 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-11-01 21:01
Hello Xavier,

> select (an poll) returns a read event and an exceptional condition 
> for the socket, but there is no normal data to read, only out-of-band
> data.

That's against POSIX:
"""
POLLIN
Data other than high-priority data may be read without blocking. For STREAMS, this flag is set in revents even if the message is of zero length. 
"""

And indeed, that's a known kernel regression introduced in 2.6.28, and fixed by this commit:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=b634f87522dff87712df8bda2a6c9061954d552a
http://kerneltrap.org/mailarchive/linux-netdev/2010/3/15/6271951

Note that there might still be a problem with the current code:
recv() can return EAGAIN on a FD reported readable/writable by select() (for example if the network stack received an input packet and then discards it because of an invalid checksum, or because the output socket buffer has room left but not enough to accomodate the packet we're trying to send): I'll have to think a bit to see if we can do something about this, but that's another issue.
Closing as invalid.
msg146836 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2011-11-02 10:29
Hi Charles-François,

> And indeed, that's a known kernel regression introduced in 2.6.28,
> and fixed by this commit:
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=b634f87522dff87712df8bda2a6c9061954d552a
> http://kerneltrap.org/mailarchive/linux-netdev/2010/3/15/6271951

The BlockingIOError exception occurs on linux 2.6.30 for me.


> Note that there might still be a problem with the current code:
> recv() can return EAGAIN on a FD reported readable/writable by
> select() (for example if the network stack received an input packet
> and then discards it because of an invalid checksum, or because the
> output socket buffer has room left but not enough to accomodate the
> packet we're trying to send): I'll have to think a bit to see if we
> can do something about this, but that's another issue.

It is not clear why recv() can return EAGAIN because we're trying to
_send_ a packet ;-)


> Closing as invalid

Ok, 2.6.30 is an old kernel and urgent data is mostly never used.
msg146838 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-11-02 14:17
> It is not clear why recv() can return EAGAIN because we're trying to
> _send_ a packet ;-)

Well, I was refering to a problem with the current implementation, not
limited to this specific example.

>> Closing as invalid
>
> Ok, 2.6.30 is an old kernel and urgent data is mostly never used.

I'll update test_asyncore to actually call recv(MSG_OOB), it's
probably a good idea to check that (hopefuly it won't break on OS
X...).
History
Date User Action Args
2011-11-02 14:17:38neologixsetmessages: + msg146838
2011-11-02 10:29:08xdegayesetmessages: + msg146836
2011-11-01 21:01:05neologixsetstatus: open -> closed

nosy: + neologix
messages: + msg146806

resolution: not a bug
stage: resolved
2011-11-01 16:04:39xdegayecreate