classification
Title: select.poll.poll fails on BSDs with arbitrary negative timeouts
Type: behavior Stage: resolved
Components: Extension Modules, FreeBSD, macOS Versions: Python 3.7, Python 3.6, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Riccardo Coccioli, koobs, ned.deily, ronaldoussoren, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2017-09-03 22:21 by Riccardo Coccioli, last changed 2017-10-18 12:05 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3277 merged Riccardo Coccioli, 2017-09-03 22:38
PR 4033 merged python-dev, 2017-10-18 08:14
PR 4034 merged Riccardo Coccioli, 2017-10-18 11:24
Messages (7)
msg301202 - (view) Author: Riccardo Coccioli (Riccardo Coccioli) * Date: 2017-09-03 22:21
According to the Python documentation for the 'poll.poll([timeout])' method in the 'select' module, any negative value for the 'timeout' parameter is valid and should have the same behaviour [1]:
    "If timeout is omitted, negative, or None, the call will block until there is an event for this poll object."

Unfortunately, unlike the Linux, on many other OSes, including, but not limited to, macOS and {Free,Open,Net}BSD, the 'poll()' system call requires that the 'timeout' parameter is a non-negative integer or exactly -1 (sometimes defined as INFTIM). Any other negative value throws an error, see [2], [3], [4] and [5].

This is a snippet of code to reproduce the error:
#-----
import select

p = select.poll()
p.poll(-100)
#-----

Expected behaviour: block until there is an event for the poll object, in this case block indefinitely
Current behaviour on macOS and FreeBSD: OSError: [Errno 22] Invalid argument

I was able to reproduce the error on:
- macOS Sierra 10.12.6 with those Python versions: 3.3.6, 3.4.6, 3.5.3, 3.6.2, 3.7.0a0 (heads/master:2ef37607b7)
- FreeBSD 11.1 with Python 3.7.0a0 (heads/master:2ef37607b7)

On Linux this doesn't happen because the 'poll()' system call accept any negative value to wait indefinitely, see [6].
To adhere with the Python documentation described behaviour, I'm sending a pull request to propose to force the 'timeout' value passed to the 'poll()' system call to be exactly -1 (or INFTIM where defined) when a negative value is given.
This will not change the current behaviour on Linux and will have the behaviour described in the documentation on other OSes where is currently failing with an error.

[1] https://docs.python.org/3/library/select.html#poll-objects
[2] https://www.freebsd.org/cgi/man.cgi?poll
[3] https://man.openbsd.org/poll.2
[4] http://netbsd.gw.com/cgi-bin/man-cgi/man?poll
[5] From macOS 'man poll': "If timeout is greater than zero, it specifies a maximum interval (in milliseconds) to wait for any file descriptor to become ready. If timeout is zero, then poll() will return without blocking. If the value of timeout is -1, the poll blocks indefinitely."
[6] http://man7.org/linux/man-pages/man2/poll.2.html
msg301308 - (view) Author: Riccardo Coccioli (Riccardo Coccioli) * Date: 2017-09-05 13:08
This can actually be reproduced with Python 2.7 too (thanks @thiell to let me know). At first I thought that it was not because it doesn't repro with the stock macOS-shipped Python 2.7.10 on macOS Sierra 10.12.6, where the select.poll() is not available at all, see below.

Updated list of version where I was able to reproduce the error:
- macOS Sierra 10.12.6 with those Python versions: 2.7.10, 2.7.13, 3.3.6, 3.4.6, 3.5.3, 3.6.2, 3.7.0a0 (heads/master:2ef37607b7)
- FreeBSD 11.1 with those Python versions: 2.7.13, 3.7.0a0 (heads/master:2ef37607b7)

For reference, the repro code executed with the stock macOS-shipped Python:
#------
$ /usr/bin/python
Python 2.7.10 (default, Feb  7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import select
>>> p = select.poll()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'poll'
#------

If the PR 3277 that I've sent against the master branch with the fix will be accepted, I'm ready to send additional PRs to backport the fix in all affected versions.
msg304472 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-10-16 13:00
Shouldn't epoll_poll() be fixed too? Only -1 is documented as an infinity timeout in epoll_wait (2).
msg304473 - (view) Author: Riccardo Coccioli (Riccardo Coccioli) * Date: 2017-10-16 13:19
Although it's documented as -1 in Linux man page [1], from my quick tests I was not able to get any error with negative values different from -1 and it seems to wait indefinitely as expected. Looking also at its implementation in [2], it doesn't seem to differentiate between negative values. It could be argued that is implementation dependent at the moment and the behaviour might change in the future.
But, on a related note, the Python documentation doesn't say much either as what are acceptable values for the timeout parameter, see [3].

So at the moment there isn't any discrepancy between the Python documentation and the current behaviour IMHO, but I'm happy to open a separate task and send a PR if you think this should be improved/fixed too.

[1] http://man7.org/linux/man-pages/man2/epoll_wait.2.html
[2] http://elixir.free-electrons.com/linux/latest/source/fs/eventpoll.c#L1754
[3] https://docs.python.org/3.7/library/select.html
msg304517 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-10-17 19:45
New changeset 6cfa927ceb931ad968b5b03e4a2bffb64a8a0604 by Victor Stinner (Riccardo Coccioli) in branch 'master':
bpo-31334: Fix timeout in select.poll.poll() (GH-3277)
https://github.com/python/cpython/commit/6cfa927ceb931ad968b5b03e4a2bffb64a8a0604
msg304575 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-10-18 12:04
New changeset 27b951c63353345cdf7a9a8c4c8133a5dafd6a80 by Serhiy Storchaka (Riccardo Coccioli) in branch '2.7':
[2.7] bpo-31334: Fix timeout in select.poll.poll() (GH-3277) (#4034)
https://github.com/python/cpython/commit/27b951c63353345cdf7a9a8c4c8133a5dafd6a80
msg304576 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-10-18 12:05
New changeset 97abcabc195b87d6a5562dbb867a469fac27d3f6 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31334: Fix timeout in select.poll.poll() (GH-3277) (#4033)
https://github.com/python/cpython/commit/97abcabc195b87d6a5562dbb867a469fac27d3f6
History
Date User Action Args
2017-10-18 12:05:18serhiy.storchakasetmessages: + msg304576
2017-10-18 12:05:03serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-10-18 12:04:08serhiy.storchakasetmessages: + msg304575
2017-10-18 11:24:06Riccardo Cocciolisetpull_requests: + pull_request4008
2017-10-18 08:14:33python-devsetkeywords: + patch
pull_requests: + pull_request4007
2017-10-17 19:45:09vstinnersetmessages: + msg304517
2017-10-16 13:19:50Riccardo Cocciolisetmessages: + msg304473
2017-10-16 13:04:57vstinnersetnosy: + vstinner
2017-10-16 13:00:43serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg304472
2017-10-14 07:30:29serhiy.storchakasetstage: patch review
versions: - Python 3.3, Python 3.4, Python 3.5
2017-09-05 13:08:39Riccardo Cocciolisetmessages: + msg301308
versions: + Python 2.7
2017-09-03 22:38:40Riccardo Cocciolisetpull_requests: + pull_request3320
2017-09-03 22:21:42Riccardo Cocciolicreate