classification
Title: OSX broken poll testing doesn't work
Type: behavior Stage:
Components: Library (Lib), Macintosh Versions: Python 3.2, Python 3.1, Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: ronaldoussoren Nosy List: bbangert, flox, foom, giampaolo.rodola, loewis, nicdumz, ronaldoussoren
Priority: normal Keywords: buildbot, patch

Created on 2009-02-04 22:32 by bbangert, last changed 2011-05-07 08:13 by ronaldoussoren. This issue is now closed.

Files
File name Uploaded Description Edit
issue5154.patch ronaldoussoren, 2011-03-16 14:04 review
Messages (11)
msg81171 - (view) Author: Ben Bangert (bbangert) Date: 2009-02-04 22:32
OSX 10.5 has a broken poll, this is an already known issue as I noticed
there's even a HAVE_BROKEN_POLL option that is in the source:
http://mail.python.org/pipermail/python-checkins/2006-April/051710.html

However, this run-time check apparently does not work, as I continue to
get socket errors related to the broken poll (On *any* version of Python
I compile from 2.4 through 2.6) unless I recompile Python and manually
comment out HAVE_POLL like the first line of Apple's Python patch here:
http://www.opensource.apple.com/darwinsource/10.5.6/python-30.1.2/fix/pyconfig.ed

Further, I think its quite likely that other bugs being submitted all
over the place from users running MacPorts on OSX are misdiagnosing this
bug as they believe its related to other software, when it is in fact
specific to custom compiles (MacPorts included) of Python. This bug
usually manifests itself as a Socket 35 error like so:
http://bugs.python.org/issue1085
http://www.cherrypy.org/ticket/598

This issue was diagnosed for me by Phil Jenvey, who believes that all
versions of OSX to date have had a broken poll (which reflects my paste
experience with this socket 35 bug as well).

I'm not sure of the most appropriate fix, I've submitted a bug to
MacPorts as well with the recommendation that for now they apply a patch
to undef HAVE_POLL, as Apple does.
msg81174 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-02-04 22:39
In what specific way is poll() broken?
msg81175 - (view) Author: Ben Bangert (bbangert) Date: 2009-02-04 22:41
I don't know specifically, Phil Jenvey probably would as he's the one
that told me it is. Apple apparently knows this an turns it off in their
version of Python that is supplied.
msg81176 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-02-04 22:48
> I don't know specifically, Phil Jenvey probably would as he's the one
> that told me it is. Apple apparently knows this an turns it off in their
> version of Python that is supplied.

That's not a sufficient reason to copy their work-around. A problem must
be understood fully before any solution is attempted. I'm -1 on
work-arounds.
msg81178 - (view) Author: Ben Bangert (bbangert) Date: 2009-02-04 22:58
Ah, sorry, misunderstanding. I'm not recommending that Python copy
Apple's patch. I was pointing out that the HAVE_BROKEN_POLL test
apparently doesn't work, and that *all* manually compiled copies of
Python done on OSX will suffer this socket 35 error without having
HAVE_POLL commented out.

As I mention in the original message, I'm not sure of the most
appropriate fix. :)
msg83156 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-03-04 22:54
poll(3) doesn't work for all types of filedescriptors on OSX.

Specifically:

  BUGS
     The poll() system call currently does not support devices.


(That's from the manpage of poll). This is why Apple doesn't expose 
select.poll in their build of Python.

I haven't used select.poll myself and therefore don't know if the issues 
with the poll system call are serious enough to warrant hiding 
select.poll on OSX.
msg111992 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-07-29 19:43
Time to try to get this going again...

What exactly is wrong with poll on OSX? Both the configure test and the runtime test for poll behavior work as designed.  

According to the first message there are other problems with poll on OSX, but without a clear description of what is wrong and how to reproduce this it will be impossible to do anything about it.

As noted in msg83156 Apple removes socket.poll from their builds because poll(3) doesn't work for all types of file descriptors (in particular not for devices), and I don't know if that is a serious enough problem to disable socket.poll in our builds as well.
msg111993 - (view) Author: James Y Knight (foom) Date: 2010-07-29 20:17
The reason it's a problem is because a "device" is everything other than a socket, pipe, slave-side of tty, or file. That is, /dev/null, /dev/zero, /dev/tty, psuedo-tty masters from openpty (e.g. for running subprocesses), etc.

I find it quite amazing that this is still broken after so long. Would it really be that hard for them to unify the select() and poll() code in the kernel? Sigh.

BTW, this same issue also affects kqueue.

But I'd say that Apple's utter failure in writing a working kernel is not really enough reason to disable the poll call for Python. If people want to use it only for the kinds of files that it works for, I don't see why python should prevent that.

Note that in the distant past, poll on OSX was *severely* broken, as it was a userspace wrapper over select(). In those days, disabling it was certainly the right thing to do.
msg114003 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-08-15 19:33
Same issue, probably, on buildbot "PPC Tiger"

======================================================================
ERROR: test_main (test.test_signal.InterProcessSignalTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.1.parc-tiger-1/build/Lib/test/test_signal.py", line 156, in test_main
    child = os.fork()
OSError: [Errno 35] Resource temporarily unavailable


http://www.python.org/dev/buildbot/all/builders/PPC%20Tiger%203.1/builds/246

And similar output on 2.6 and 3.x as well.
msg130705 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2011-03-12 23:10
Florent: I don't understand why the traceback you show is related to this issue.
msg131116 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2011-03-16 14:04
I propose to close this as won't fix. The reason: poll on OSX does work, but fairly limited in the kinds of file descriptors it works with. The HAVE_BROKEN_POLL test is meant to detect poll implementation that don't even manage to implement the API correctly.

The attached patch adds notes to the documentation to warn users about the limitations of poll and kqueue on MacOSX.
History
Date User Action Args
2011-05-07 08:13:45ronaldoussorensetstatus: open -> closed
resolution: wont fix
2011-03-16 14:04:56ronaldoussorensetfiles: + issue5154.patch

messages: + msg131116
keywords: + patch
nosy: loewis, ronaldoussoren, foom, giampaolo.rodola, nicdumz, bbangert, flox
2011-03-12 23:10:39ronaldoussorensetnosy: loewis, ronaldoussoren, foom, giampaolo.rodola, nicdumz, bbangert, flox
messages: + msg130705
2010-08-15 19:33:29floxsetnosy: + flox
messages: + msg114003

assignee: ronaldoussoren
components: + Macintosh
keywords: + buildbot
2010-08-04 22:55:26terry.reedysetversions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 2.5, Python 2.4, Python 3.0
2010-07-29 20:17:26foomsetnosy: + foom
messages: + msg111993
2010-07-29 19:43:29ronaldoussorensetmessages: + msg111992
2009-04-14 17:46:21NicDumZsetnosy: + nicdumz
2009-03-04 23:28:01giampaolo.rodolasetnosy: + giampaolo.rodola
2009-03-04 22:54:16ronaldoussorensetmessages: + msg83156
2009-03-04 22:28:28ronaldoussorensetnosy: + ronaldoussoren
2009-02-04 22:58:50bbangertsetmessages: + msg81178
2009-02-04 22:48:16loewissetmessages: + msg81176
2009-02-04 22:41:57bbangertsetmessages: + msg81175
2009-02-04 22:39:57loewissetnosy: + loewis
messages: + msg81174
2009-02-04 22:32:21bbangertcreate