classification
Title: macOS 12 poll syscall returns prematurely
Type: Stage: commit review
Components: macOS, Tests Versions: Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: MicroTransactionsMatterToo, gregory.p.smith, ncoghlan, ned.deily, rbcollins, ronaldoussoren, vstinner
Priority: normal Keywords: patch

Created on 2016-09-11 23:36 by MicroTransactionsMatterToo, last changed 2017-08-10 23:34 by vstinner.

Files
File name Uploaded Description Edit
buildlog.log MicroTransactionsMatterToo, 2016-09-11 23:36 Build Log
eintr_tester.diff MicroTransactionsMatterToo, 2016-09-12 00:09
issue28087.patch ned.deily, 2016-09-12 02:02
smime.p7s MicroTransactionsMatterToo, 2016-09-12 02:03
smime.p7s MicroTransactionsMatterToo, 2016-09-12 02:09
Pull Requests
URL Status Linked Edit
PR 462 merged ned.deily, 2017-03-04 10:58
PR 463 merged ned.deily, 2017-03-04 11:18
PR 973 merged vstinner, 2017-04-03 11:27
PR 1424 merged vstinner, 2017-05-03 09:31
PR 1426 closed vstinner, 2017-05-03 09:59
PR 1664 closed vstinner, 2017-05-19 06:05
Messages (33)
msg275887 - (view) Author: Ennis Massey (MicroTransactionsMatterToo) Date: 2016-09-11 23:36
Upon calling the select.poll objects poll method with no file descriptors registered, it will return an empty list immediately, regardless of the timeout given. This is a problem in the Mac OS X kernels implementation of the poll syscall, but the test can be easily fixed by binding a throwaway socket to the poll objects, and then using the poll method as usual.

Build details are attached
msg275889 - (view) Author: Ennis Massey (MicroTransactionsMatterToo) Date: 2016-09-11 23:52
Patch exemplifying a fix for this in the test files
msg275894 - (view) Author: Ennis Massey (MicroTransactionsMatterToo) Date: 2016-09-12 00:09
Fixed patch file, now in diff format
msg275912 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-09-12 02:02
Thanks for the report.  This seems to be a problem visible in the public betas of the upcoming macOS 12 release, not in previous releases (10.11 and earlier).  Do you know if the problem has been reported to Apple?  Also, I've uploaded a reviewable version of your patch.
msg275913 - (view) Author: Ennis Massey (MicroTransactionsMatterToo) Date: 2016-09-12 02:03
Apple doesn’t allow access to other peoples bug reports, so I couldn’t say whether it’s been reported.

Many Thanks,
Ennis Massey

Wet Ferret Studios <http://wet-ferret-studios.azurewebsites.net/>
msg275914 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-09-12 02:09
OK, if you get a chance, it would be helpful to submit a RADAR on it and update this issue.  I neglected to mention that your patch does seem to work, e.g., test_eintr no longer fails.
msg275915 - (view) Author: Ennis Massey (MicroTransactionsMatterToo) Date: 2016-09-12 02:09
Ok, cool

Many Thanks,
Ennis Massey

Wet Ferret Studios <http://wet-ferret-studios.azurewebsites.net/>
msg275929 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2016-09-12 03:06
@Ned - any objection to my committing this at this point?
msg275932 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-09-12 03:09
I'm not sure it should be a permanent fix.  Perhaps you could add the test decorator so that it is only tested on 10.12.  I don't have time to look at it more myself before b1.
msg278623 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-10-13 22:58
(From https://github.com/curl/curl/issues/1057, the curl project has also seen this and opened an issue with Apple against macOS 10.12, RADAR 28372390.)
msg288055 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-02-18 05:00
The failures in test_eintr and test_asyncio are confusing new contributors on Mac OS X - could we get some variant of this workaround merged until Apple fix their bug?
msg288070 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-02-18 09:48
Sorry, I meant test_poll and test_asyncore as described in http://bugs.python.org/issue28456 (I misremembered due to the test_poll failure mentioning EINTR)
msg290300 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-03-24 22:55
New changeset 1d391f926b37484b8d4b326003a72c0084db19ec by Ned Deily in branch '3.6':
[3.6] bpo-28087: Skip test_asyncore and test_eintr poll failures on macOS (#463)
https://github.com/python/cpython/commit/1d391f926b37484b8d4b326003a72c0084db19ec
msg290301 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-03-24 22:56
New changeset de04644627f82d9dc48b3423def7ff5b4aa1926a by Ned Deily in branch 'master':
bpo-28087: Skip test_asyncore and test_eintr poll failures on macOS. (#462)
https://github.com/python/cpython/commit/de04644627f82d9dc48b3423def7ff5b4aa1926a
msg291065 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-03 11:29
test_asyncore also fails on Python 2.7 on macOS Sierra. So I proposed a backport: see my PR 973.

======================================================================
FAIL: test_handle_expt (test.test_asyncore.TestAPI_UsePoll)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/2.7.billenstein-sierra/build/Lib/test/test_asyncore.py", line 620, in test_handle_expt
    self.loop_waiting_for_flag(client)
  File "/Users/buildbot/buildarea/2.7.billenstein-sierra/build/Lib/test/test_asyncore.py", line 519, in loop_waiting_for_flag
    self.fail("flag not set")
AssertionError: flag not set

http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/2/steps/test/logs/stdio
msg291078 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-03 16:11
New changeset 23d6eb656ec29140fcca1c9261b7953e2312b171 by Victor Stinner in branch '2.7':
bpo-28087: Skip test_asyncore and test_eintr poll failures on macOS. (#462) (#973)
https://github.com/python/cpython/commit/23d6eb656ec29140fcca1c9261b7953e2312b171
msg291082 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-03 16:50
Ah! Python 2.7 tests succeeded on Sierra!

http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/3
msg291137 - (view) Author: Ennis Massey (MicroTransactionsMatterToo) Date: 2017-04-04 20:17
Makes sense it would also fail. They both use the same syscall. Gud job on the backport

Sent from my iPhone

> On 4/04/2017, at 4:50 AM, STINNER Victor <report@bugs.python.org> wrote:
> 
> 
> STINNER Victor added the comment:
> 
> Ah! Python 2.7 tests succeeded on Sierra!
> 
> http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/3
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28087>
> _______________________________________
msg292869 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-03 09:41
Instead of skipping the test, it would be safer to not provide select.poll() if it's known to be broken. Curl added a check in configure:
https://github.com/curl/curl/commit/9297ca49f5f3caca938a679b9c1feeb719e61ddb

The problem is that I understood that Python provides a single binary for all supported macOS versions. So if we go for a check, it should be done as runtime, as the current select_have_broken_poll() test written for macOS.

On macOS without the bug, curl check takes 0.5 second which is not ideal :-/

Would it be possible to design a test which doesn't block?
msg292870 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-03 09:43
> Would it be possible to design a test which doesn't block?

If there is no obvious way to detect poll() functionnaly, a workaround is to get the Darwin version from uname() and blacklist macOS versions known to have the bug (disable select.poll() on these versions).
msg292875 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-03 10:04
https://github.com/python/cpython/pull/1426 implements the blacklist option.

I chose the conservative approach: hope that poll() will be fixed in 16.6.x. The problem of this approach is that test_asyncore and test_eintr will fail again if Apple doesn't fix poll().

Since I would like to backport this fix in 2.7, 3.5 and 3.6 and not have to modify the code at each broken macOS release, maybe we can use the pessimistic solution: blacklist Darwin >= 16.x, and only start again to whitelist once Apple releases a fixed macOS.
msg292876 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-03 10:04
New changeset 23b312b087582cec5cc8c528d267eb42061601ac by Victor Stinner in branch '3.5':
[3.6] bpo-28087: Skip test_asyncore and test_eintr poll failures on macOS (#463) (#1424)
https://github.com/python/cpython/commit/23b312b087582cec5cc8c528d267eb42061601ac
msg293168 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-05-06 19:35
Please DO NOT REMOVE select.poll() on MacOS.  It is still useful.

Apple needs to live with their bug that they refuse to acknowledge by forcing developers to learn the special case that it now breaks in (0 fds).  We should not pretend that poll() does not exist as a result.

For reference about the details of the MacOS change they were stupid enough to force upon the world, see this comment - which links to the source of their bug in the "open" source side of their OS.
  https://github.com/curl/curl/issues/1057#issuecomment-252286017
msg293169 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-05-06 19:39
The only thing CPython should do is apply workarounds to any standard library internal uses of poll with 0 fds guarded by a MacOS platform check.
msg293176 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-05-07 00:17
Do we need to work around this issue at all?

Apple appears to have fixed it already:
  https://github.com/opensource-apple/xnu/commit/0cccba1c831cfe36f0d67d672f8aa67b4a7d4b6a#diff-e61c2932bb9d5cea2dd0732acd8ec626R1783

A comment in https://github.com/curl/curl/issues/1057 suggests Apple has shipped that change in 10.12.2.

If you still want to have Python deal with the 10.12.0 and 10.12.1 cases you could update your PR to check those versions, but given these are security fix releases as 10.12.4 is already current I think it is better to tell people just to apply their security updates and not bother with a workaround in Python.
msg293186 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-07 11:35
You want to get poll() on macOS. I'm ok with that but I would like to see
tests for it. We should remove the skip in the test.

Ok, I will update my PR to just blacklist macOS 10.12.0 and 10.2.1, and
reenable the test. I just have to find the Darwin versions.

I don't think that we have to suggests Python users to upgrade, I expect
macOS popups requesting to apply upgrades are enough ;-)
msg293944 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-05-19 00:02
If the problem is fixed in 10.12.2, I agree with gps that there is no need to add a workaround for 10.12.0 and .1.  We should only ever need to support the most recent macOS point release; it is the user's responsibility to keep up-to-date and with most users these days that happens automatically.
msg293951 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-19 06:06
I abandonned PR 1426 which proposed to blacklist bogus macOS versions, and instead I proposed a new PR to simply reenable previsouly skipped tests (because of the bug which is now fixed).
https://github.com/python/cpython/pull/1664

I'm unable to test my own PR right now. Ned Deily: would you mind to test it and review my change please?
msg294269 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-05-23 17:57
Unfortunately, with the tests reenabled, they still fail with the most recent release of macOS 10.12 (10.12,5) so perhaps the curl issue was a different problem?
msg294277 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-23 19:14
Ned Deily added the comment:
> Unfortunately, with the tests reenabled, they still fail with the most recent release of macOS 10.12 (10.12,5)

Oh. In that case, I will update my patch to blacklist macOS versions
with the bug (remove select.poll() depending on macOS version).
msg294304 - (view) Author: Ennis Massey (MicroTransactionsMatterToo) Date: 2017-05-24 01:00
Actually, having managed to get macOS to update, it seems to be odd, as it correctly delays. Not sure what’s going on there, or whether it’s my setup. I’ll look into it some more. I’m also thinking of writing/finding a set of tests for core POSIX stuff like this, so we can avoid Apple screwing stuff up like they have.
Many Thanks,
Ennis Massey
ennisbaradine@gmail.com
msg294309 - (view) Author: Ennis Massey (MicroTransactionsMatterToo) Date: 2017-05-24 02:06
Well, the tests worked on macOS 10.12.6 beta, although my school network broke the urllib tests
Many Thanks,
Ennis Massey
ennisbaradine@gmail.com
msg300128 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-08-10 23:34
What is the status of this issue? Is there still something to do?

If you ask me my opinion, I would just suggest to remove select.poll() on macOS to stop to have to bother with poll() bugs which only trigger at runtime :-/
History
Date User Action Args
2017-08-10 23:34:58vstinnersetmessages: + msg300128
versions: - Python 3.5
2017-05-24 02:06:12MicroTransactionsMatterToosetmessages: + msg294309
2017-05-24 01:00:33MicroTransactionsMatterToosetmessages: + msg294304
2017-05-23 19:14:46vstinnersetmessages: + msg294277
2017-05-23 17:57:44ned.deilysetmessages: + msg294269
2017-05-19 06:06:55vstinnersetmessages: + msg293951
2017-05-19 06:05:41vstinnersetpull_requests: + pull_request1759
2017-05-19 00:02:29ned.deilysetmessages: + msg293944
2017-05-07 11:35:22vstinnersetmessages: + msg293186
2017-05-07 00:17:48gregory.p.smithsetmessages: + msg293176
versions: + Python 2.7
2017-05-06 19:39:39gregory.p.smithsetmessages: + msg293169
2017-05-06 19:35:18gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg293168
2017-05-03 10:05:00vstinnersetmessages: + msg292876
2017-05-03 10:04:15vstinnersetmessages: + msg292875
2017-05-03 09:59:03vstinnersetpull_requests: + pull_request1531
2017-05-03 09:43:00vstinnersetmessages: + msg292870
2017-05-03 09:41:40vstinnersetmessages: + msg292869
2017-05-03 09:36:30vstinnerlinkissue30239 superseder
2017-05-03 09:31:12vstinnersetpull_requests: + pull_request1530
2017-04-04 20:17:32MicroTransactionsMatterToosetmessages: + msg291137
2017-04-03 16:50:04vstinnersetmessages: + msg291082
2017-04-03 16:11:24vstinnersetmessages: + msg291078
2017-04-03 11:29:11vstinnersetnosy: + vstinner
messages: + msg291065
2017-04-03 11:27:54vstinnersetpull_requests: + pull_request1150
2017-04-01 05:48:41serhiy.storchakasetpull_requests: - pull_request1036
2017-03-31 16:36:30dstufftsetpull_requests: + pull_request1036
2017-03-24 22:56:07ned.deilysetmessages: + msg290301
2017-03-24 22:55:14ned.deilysetmessages: + msg290300
2017-03-04 11:18:15ned.deilysetpull_requests: + pull_request386
2017-03-04 10:58:59ned.deilysetpull_requests: + pull_request385
2017-02-18 09:48:34ncoghlansetmessages: + msg288070
2017-02-18 05:00:33ncoghlansetnosy: + ncoghlan
messages: + msg288055
2016-10-16 23:24:11ned.deilylinkissue28456 superseder
2016-10-13 22:58:13ned.deilysetmessages: + msg278623
2016-09-12 03:09:32ned.deilysetmessages: + msg275932
2016-09-12 03:06:58rbcollinssetstage: patch review -> commit review
2016-09-12 03:06:50rbcollinssetnosy: + rbcollins
messages: + msg275929
2016-09-12 02:09:36MicroTransactionsMatterToosetfiles: + smime.p7s

messages: + msg275915
2016-09-12 02:09:04ned.deilysetmessages: + msg275914
2016-09-12 02:03:43MicroTransactionsMatterToosetfiles: + smime.p7s

messages: + msg275913
2016-09-12 02:02:06ned.deilysetfiles: + issue28087.patch

components: + Tests
title: Mac OS X poll syscall returns prematurely -> macOS 12 poll syscall returns prematurely
type: compile error ->
versions: + Python 3.5, Python 3.7
messages: + msg275912
stage: patch review
2016-09-12 00:09:21MicroTransactionsMatterToosetfiles: + eintr_tester.diff
keywords: + patch
messages: + msg275894
2016-09-12 00:07:45MicroTransactionsMatterToosetfiles: - eintr_tester.py
2016-09-11 23:52:04MicroTransactionsMatterToosetfiles: + eintr_tester.py

messages: + msg275889
2016-09-11 23:36:48MicroTransactionsMatterToocreate