classification
Title: unexpected EPROTOTYPE returned by sendto on MAC OSX
Type: behavior Stage:
Components: asyncio, IO, macOS Versions: Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: ronaldoussoren Nosy List: Eamonn Nugent, asvetlov, ned.deily, racitup, ronaldoussoren, yselivanov
Priority: normal Keywords:

Created on 2018-05-09 19:04 by racitup, last changed 2019-07-14 14:03 by ronaldoussoren.

Messages (3)
msg316328 - (view) Author: Richard C (racitup) Date: 2018-05-09 19:04
The following exception is raised unexpectedly on macOS versions 10.13, 10.12 & 10.11 at least. It appears to be macOS specific (works okay on Linux).

Further information can be found at the following links:
https://github.com/benoitc/gunicorn/issues/1487
http://erickt.github.io/blog/2014/11/19/adventures-in-debugging-a-potential-osx-kernel-bug/

[2017-03-20 00:46:39 +0100] [79068] [ERROR] Socket error processing request.
Traceback (most recent call last):
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 66, in handle
    six.reraise(*sys.exc_info())
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/six.py", line 625, in reraise
    raise value
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 56, in handle
    self.handle_request(listener_name, req, client, addr)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/ggevent.py", line 152, in handle_request
    super(GeventWorker, self).handle_request(*args)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 129, in handle_request
    six.reraise(*sys.exc_info())
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/six.py", line 625, in reraise
    raise value
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 115, in handle_request
    resp.write(item)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/http/wsgi.py", line 362, in write
    util.write(self.sock, arg, self.chunked)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/util.py", line 321, in write
    sock.sendall(data)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gevent-1.2.1-py3.6-macosx-10.12-x86_64.egg/gevent/_socket3.py", line 418, in sendall
    data_sent += self.send(data_memory[data_sent:], flags)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gevent-1.2.1-py3.6-macosx-10.12-x86_64.egg/gevent/_socket3.py", line 391, in send
    return _socket.socket.send(self._sock, data, flags)
OSError: [Errno 41] Protocol wrong type for socket
msg316353 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2018-05-10 09:50
The second link contains an explanation of what's going on, and that this is unexpected behaviour of the macOS kernel.

I'm not sure what we could do about this, the blog post explains that this error can happen when send(2) is called while the socket is teared down. A possible workaround (based on reading the blog post and without fully analysing the side effects) is to treat EPROTOTYPE the same as EGAIN in the python wrappers for send/write/sendto. 

That's easier said than done though, socketmodule.c indirectly calls socket functions through a helper function that does some bookkeeping (including handling EGAIN) and is used for more than just the send calls.
msg327724 - (view) Author: Eamonn Nugent (Eamonn Nugent) Date: 2018-10-15 01:34
Fwiw, this also happens in asyncio. Theoretically, I *think* you could wrap a try-except as a monkey patch.

My stack trace is:

Fatal write error on socket transport
protocol: <RequestHandler connected>
transport: <_SelectorSocketTransport fd=163 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 761, in write
    n = self._sock.send(data)
OSError: [Errno 41] Protocol wrong type for socket

I can try to PR the fix mentioned by @ronaldoussoren, though it might take me a bit. I'm happy to do whatever to get this fixed, though, since it's flooding my terminal with several hundred errors every time it happens...
History
Date User Action Args
2019-07-14 14:03:59ronaldoussorensetassignee: ronaldoussoren
2018-10-15 22:45:35ned.deilysetnosy: + yselivanov, asvetlov
components: + asyncio
2018-10-15 01:34:15Eamonn Nugentsetnosy: + Eamonn Nugent
messages: + msg327724
2018-05-10 09:50:58ronaldoussorensetmessages: + msg316353
2018-05-09 19:04:59racitupcreate