classification
Title: Selector.select() hangs when there is nothing to select
Type: Stage: resolved
Components: asyncio Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Aleksey Kladov, gvanrossum, martin.panter, neologix, russelldavis, yselivanov
Priority: normal Keywords: patch

Created on 2015-11-20 14:18 by Aleksey Kladov, last changed 2020-04-15 20:06 by gvanrossum.

Pull Requests
URL Status Linked Edit
PR 19508 merged russelldavis, 2020-04-14 08:12
Messages (13)
msg254975 - (view) Author: Aleksey Kladov (Aleksey Kladov) Date: 2015-11-20 14:18
The following hangs on Linux

```Python
>>> import selectors
>>> s = selectors.DefaultSelector()
>>> s.select()
```

On Mac it returns an empty list.
msg254977 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-11-20 14:36
Your code doesn't make sense. You listen to 0 file descriptor and wait forever until you get events, which will never occur. I'm not even sure that it should be allowed by the Selector API.

What's the point of your bug report? Does your application really rely on the behaviour of the selector when no file descriptor is registered?

The behaviour changes between Linux and OS X because the "default" selector is a different selector on Linux (epoll) and OS X (kqueue? select?)
msg255060 - (view) Author: Aleksey Kladov (Aleksey Kladov) Date: 2015-11-21 16:01
>What's the point of your bug report?

To show surprising platform-dependent API behavior. I don't know what is the correct behavior here, but it should be cross platform. Seems like the most sane option is to throw an exception. 


>Does your application really rely on the behaviour of the selector when no file descriptor is registered?

It was developed on Mac and relied (quite probably incorrectly) on the empty list result. When I run it on linux, it unexpectedly hanged.

So, I would like to either

* observe an exception on all platforms
* observe an empty list on all platforms
* observe infinite blocking on all platforms

I think this special case should also be mentioned in the docs.
msg255081 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-22 02:12
I would expect it to hang (until the timeout expired) of no files are added. If it returns an empty list, that should mean the timeout expired.

I guess Mac uses KqueueSelector. I would certainly expect SelectSelector to hang. Maybe the underlying kqueue implementation behaves differently though. Perhaps Python should handle this as a special case in the selectors module?
msg255100 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-11-22 13:52
Aleksey Kladov: "To show surprising platform-dependent API behavior."

Hum ok, it makes sense :-) I agree that it would be better to have a portable behaviour in the selectors module which is supposed to be an abstraction providing a portable behaviour. We already wrote code to have the same behaviour when a file descriptor is closed (handle EBADF).

Aleksey Kladov: "I think this special case should also be mentioned in the docs."

I disagree, we can do better than documenting "bugs" :-)

select.select([], [], [], None) raise an OSError "An invalid argument was supplied" on Windows. I propose to use this behaviour on all platforms for all selectors.

I should be easy to write a patch in selectors, is anyone interested to write a patch?
msg255101 - (view) Author: Aleksey Kladov (Aleksey Kladov) Date: 2015-11-22 14:27
>I disagree, we can do better than documenting "bugs" :-)

The result of `select.select([], [], [], None)` should defintelly be documented. 

Three platoforms made three different decisions in similar situation, so any behavior is reasonable and it should be made clear which one is used by Python.
msg255116 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2015-11-22 23:01
I don't think we should "fix" the select module; it's advertised as a wrapper around the various syscalls, and if the platforms disagree on what that syscall (e.g. select()) does we shouldn't try to fix it. (Documenting the differences is fine.)

However the *selectors* module is a different thing. It defines a better abstraction that's supposedly independent from the underlying syscall. I agree with Aleksey that it should just hang when no FDs are registered and no timeout is given -- this is the only logical extension of its behavior when a FD is registered or a timeout is given. This could be used to wait until e.g. a signal arrives.

It's illogical that it would behave differently if a FD was registered that will never fire, or if a timeout of a billion seconds was given (although that may be an easy hack to wait forever if the underlying syscall doesn't like this).

The asyncio package would probably be broken because of this except that it always has a FD registered (the self-pipe).
msg366163 - (view) Author: Russell Davis (russelldavis) * Date: 2020-04-10 22:55
Looks like this is caused by #29255
msg366188 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-04-11 03:46
That may well be correct. Do you want to submit a PR to fix it?
msg366533 - (view) Author: Russell Davis (russelldavis) * Date: 2020-04-15 17:52
@gvanrossum PR is ready for review: https://github.com/python/cpython/pull/19508
msg366541 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-04-15 18:57
New changeset ba1bcffe5cafc1bb0ac6fdf9ecef51e75e342707 by Russell Davis in branch 'master':
bpo-29255: Wait in KqueueSelector.select when no fds are registered (GH-19508)
https://github.com/python/cpython/commit/ba1bcffe5cafc1bb0ac6fdf9ecef51e75e342707
msg366545 - (view) Author: Russell Davis (russelldavis) * Date: 2020-04-15 19:54
I think this got auto-closed due to a link in the PR. Note that, per https://github.com/python/cpython/pull/19508#issuecomment-613317021, the behavior is still inconsistent on windows.

I think the solution there will have to be a call to sleep() when the list of fds is empty.
msg366548 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-04-15 20:06
How ironic, the other issue had to be closed manually. :-)

Reopening this one.
History
Date User Action Args
2020-04-15 20:06:54gvanrossumsetstatus: closed -> open
resolution: fixed ->
messages: + msg366548
2020-04-15 19:54:01russelldavissetmessages: + msg366545
2020-04-15 18:57:13gvanrossumsetstatus: open -> closed
resolution: fixed
messages: + msg366541

stage: patch review -> resolved
2020-04-15 17:52:15russelldavissetmessages: + msg366533
2020-04-14 08:12:25russelldavissetkeywords: + patch
stage: patch review
pull_requests: + pull_request18861
2020-04-13 22:31:37vstinnersetnosy: - vstinner
2020-04-11 03:46:33gvanrossumsetmessages: + msg366188
2020-04-10 22:55:17russelldavissetnosy: + russelldavis
messages: + msg366163
2015-11-22 23:01:15gvanrossumsetmessages: + msg255116
2015-11-22 14:27:52Aleksey Kladovsetmessages: + msg255101
2015-11-22 13:52:05vstinnersetnosy: + yselivanov, neologix, gvanrossum
messages: + msg255100
components: + asyncio
2015-11-22 02:12:07martin.pantersetnosy: + martin.panter
messages: + msg255081
2015-11-21 16:01:08Aleksey Kladovsetmessages: + msg255060
2015-11-20 14:36:38vstinnersetnosy: + vstinner
messages: + msg254977
2015-11-20 14:18:22Aleksey Kladovcreate