New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
macOS 12 poll syscall returns prematurely #72274
Comments
Upon calling the select.poll objects poll method with no file descriptors registered, it will return an empty list immediately, regardless of the timeout given. This is a problem in the Mac OS X kernels implementation of the poll syscall, but the test can be easily fixed by binding a throwaway socket to the poll objects, and then using the poll method as usual. Build details are attached |
Patch exemplifying a fix for this in the test files |
Fixed patch file, now in diff format |
Thanks for the report. This seems to be a problem visible in the public betas of the upcoming macOS 12 release, not in previous releases (10.11 and earlier). Do you know if the problem has been reported to Apple? Also, I've uploaded a reviewable version of your patch. |
Apple doesn’t allow access to other peoples bug reports, so I couldn’t say whether it’s been reported. Many Thanks, Wet Ferret Studios <http://wet-ferret-studios.azurewebsites.net/\> |
OK, if you get a chance, it would be helpful to submit a RADAR on it and update this issue. I neglected to mention that your patch does seem to work, e.g., test_eintr no longer fails. |
Ok, cool Many Thanks, Wet Ferret Studios <http://wet-ferret-studios.azurewebsites.net/\> |
@ned - any objection to my committing this at this point? |
I'm not sure it should be a permanent fix. Perhaps you could add the test decorator so that it is only tested on 10.12. I don't have time to look at it more myself before b1. |
(From curl/curl#1057, the curl project has also seen this and opened an issue with Apple against macOS 10.12, RADAR 28372390.) |
The failures in test_eintr and test_asyncio are confusing new contributors on Mac OS X - could we get some variant of this workaround merged until Apple fix their bug? |
Sorry, I meant test_poll and test_asyncore as described in http://bugs.python.org/issue28456 (I misremembered due to the test_poll failure mentioning EINTR) |
test_asyncore also fails on Python 2.7 on macOS Sierra. So I proposed a backport: see my PR 973. ====================================================================== Traceback (most recent call last):
File "/Users/buildbot/buildarea/2.7.billenstein-sierra/build/Lib/test/test_asyncore.py", line 620, in test_handle_expt
self.loop_waiting_for_flag(client)
File "/Users/buildbot/buildarea/2.7.billenstein-sierra/build/Lib/test/test_asyncore.py", line 519, in loop_waiting_for_flag
self.fail("flag not set")
AssertionError: flag not set http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/2/steps/test/logs/stdio |
Ah! Python 2.7 tests succeeded on Sierra! http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/3 |
Makes sense it would also fail. They both use the same syscall. Gud job on the backport Sent from my iPhone
|
Instead of skipping the test, it would be safer to not provide select.poll() if it's known to be broken. Curl added a check in configure: The problem is that I understood that Python provides a single binary for all supported macOS versions. So if we go for a check, it should be done as runtime, as the current select_have_broken_poll() test written for macOS. On macOS without the bug, curl check takes 0.5 second which is not ideal :-/ Would it be possible to design a test which doesn't block? |
If there is no obvious way to detect poll() functionnaly, a workaround is to get the Darwin version from uname() and blacklist macOS versions known to have the bug (disable select.poll() on these versions). |
#1426 implements the blacklist option. I chose the conservative approach: hope that poll() will be fixed in 16.6.x. The problem of this approach is that test_asyncore and test_eintr will fail again if Apple doesn't fix poll(). Since I would like to backport this fix in 2.7, 3.5 and 3.6 and not have to modify the code at each broken macOS release, maybe we can use the pessimistic solution: blacklist Darwin >= 16.x, and only start again to whitelist once Apple releases a fixed macOS. |
Please DO NOT REMOVE select.poll() on MacOS. It is still useful. Apple needs to live with their bug that they refuse to acknowledge by forcing developers to learn the special case that it now breaks in (0 fds). We should not pretend that poll() does not exist as a result. For reference about the details of the MacOS change they were stupid enough to force upon the world, see this comment - which links to the source of their bug in the "open" source side of their OS. |
The only thing CPython should do is apply workarounds to any standard library internal uses of poll with 0 fds guarded by a MacOS platform check. |
Do we need to work around this issue at all? Apple appears to have fixed it already: A comment in curl/curl#1057 suggests Apple has shipped that change in 10.12.2. If you still want to have Python deal with the 10.12.0 and 10.12.1 cases you could update your PR to check those versions, but given these are security fix releases as 10.12.4 is already current I think it is better to tell people just to apply their security updates and not bother with a workaround in Python. |
You want to get poll() on macOS. I'm ok with that but I would like to see Ok, I will update my PR to just blacklist macOS 10.12.0 and 10.2.1, and I don't think that we have to suggests Python users to upgrade, I expect |
If the problem is fixed in 10.12.2, I agree with gps that there is no need to add a workaround for 10.12.0 and .1. We should only ever need to support the most recent macOS point release; it is the user's responsibility to keep up-to-date and with most users these days that happens automatically. |
Unfortunately, with the tests reenabled, they still fail with the most recent release of macOS 10.12 (10.12,5) so perhaps the curl issue was a different problem? |
Ned Deily added the comment:
Oh. In that case, I will update my patch to blacklist macOS versions |
Actually, having managed to get macOS to update, it seems to be odd, as it correctly delays. Not sure what’s going on there, or whether it’s my setup. I’ll look into it some more. I’m also thinking of writing/finding a set of tests for core POSIX stuff like this, so we can avoid Apple screwing stuff up like they have. |
Well, the tests worked on macOS 10.12.6 beta, although my school network broke the urllib tests |
What is the status of this issue? Is there still something to do? If you ask me my opinion, I would just suggest to remove select.poll() on macOS to stop to have to bother with poll() bugs which only trigger at runtime :-/ |
No update since 2017, I close the issue. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: