Author neologix
Recipients christian.heimes, felipecruz, giampaolo.rodola, gvanrossum, meador.inge, neologix, pitrou, rosslagerwall, sbt
Date 2013-01-08.10:49:27
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAH_1eM3CwqvpPtxCkmMR58cS2KmUiFHDtQpfvbNYaoWMYR_DNA@mail.gmail.com>
In-reply-to <1357605418.38.0.653389791218.issue16853@psf.upfronthosting.co.za>
Content
So I assume that the second failure is fixed, which means that OS-X
returns a funny event (and not POLLIN/POLLHUP/POLLERR) in case of
ECONNREFUSED :-(

> 2 - In EventLoopTestsMixin::test_writer_callback if the writer socket isn't non-blocking, the test hangs forever..

This test is buggy:

"""
    def test_writer_callback(self):
        el = events.get_event_loop()
        r, w = unix_events.socketpair()
        el.add_writer(w.fileno(), w.send, b'x'*100)
        el.call_later(0.1, el.remove_writer, w.fileno())
        el.run()
        w.close()
"""

Because the writer socket isn't non-blocking, if the output socket
buffer fills up in less than 0.1 seconds, the call to w.send(b'x'*100)
will block.
select()/poll()/kqueue() use a watermark to decide whether the FD is
readable/writable: for a Unix domain socket, I guess OS-X returns that
the socket is writable is there's at least one byte free in the output
socket buffer: since send() tries to write 100 bytes at once, it
blocks.

I can reproduce the hang on Linux with vanilla (unpatched) tulip by
increasing the number of bytes sent() in one syscall:
"""
--- tulip-bf4cb136c121/tulip/events_test.py     2092-08-05
00:00:00.000000000 +0200
+++ tulip/tulip/events_test.py  2013-01-08 11:35:27.400198000 +0100
@@ -149,7 +149,7 @@
     def test_writer_callback(self):
         el = events.get_event_loop()
         r, w = unix_events.socketpair()
-        el.add_writer(w.fileno(), w.send, b'x'*100)
+        el.add_writer(w.fileno(), w.send, b'x'*(1<<18))
         el.call_later(0.1, el.remove_writer, w.fileno())
         el.run()
         w.close()
"""

(I have to do that because Linux uses an adaptive watermark, see
http://lxr.free-electrons.com/source/net/unix/af_unix.c#L2156 and
http://lxr.free-electrons.com/source/net/unix/af_unix.c#L319 ).

That's why all FDs monitored with select()/poll()/epoll() must be
non-blocking (and there's also the risk of spurious wakeups...).

> 3 - Errors:
>
> ERROR: testCreateSslTransport (tulip.events_test.PollEventLoopTests)
>   File "/Users/felipecruz/Projects/tulip3/tulip/selectors.py", line 180, in _key_from_fd
>     raise RuntimeError("No key found for fd {}".format(fd))
> RuntimeError: No key found for fd -2
>
>
> ERROR: test_sock_client_ops (tulip.events_test.KqueueEventLoopTests)
>   File "/Users/felipecruz/Projects/tulip3/tulip/selectors.py", line 180, in _key_from_fd
>     raise RuntimeError("No key found for fd {}".format(fd))
> RuntimeError: No key found for fd 77

Yes, it's the same problem: poll()/kqueue() returning garbage.
I guess I'll have to patch Selector to ignore unknown FDs, but that
really looks like an OS-X bug.
Could you dump the content of the returned kevent when the fd is not found?
History
Date User Action Args
2013-01-08 10:49:28neologixsetrecipients: + neologix, gvanrossum, pitrou, giampaolo.rodola, christian.heimes, meador.inge, rosslagerwall, sbt, felipecruz
2013-01-08 10:49:28neologixlinkissue16853 messages
2013-01-08 10:49:27neologixcreate