This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Python 2.3+ socket._fileobject handles EAGAIN with data loss
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 2.7
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, craigemery, jnelson, pitrou
Priority: normal Keywords:

Created on 2004-10-01 16:56 by jnelson, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (8)
msg60570 - (view) Author: Jon Nelson (jnelson) Date: 2004-10-01 16:56
If one has a non-blocking socket, 'sock', and uses
'sock.makefile()' to create file-like object out of it,
there exists the possibility (even probability) of data
loss in the event of an EAGAIN error.

The code that comprises _fileobject in socket does not
try to catch EAGAIN.  In the event that the read()
operation does not encounter EOF, but *has* read data
(and then gets an EAGAIN) such as when there is less
than 8KB of data to read, the socket.error traceback is
handed back to the caller of read and the read data is
discarded.  This is horrible. The _fileobject read
routine (and probably write, too) should catch
socket.error and check it for EAGAIN, and in that case,
return what it has already read (if anything). 
msg60571 - (view) Author: Jon Nelson (jnelson) Date: 2004-10-01 18:25
Logged In: YES 

An improvement would be to allow EAGAIN to surface only if
it happens as the /first/ read. 
msg114378 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-08-19 16:39
Is this still valid?
msg125275 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-04 01:33
Should work fine under 3.x.
msg192666 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-07-08 16:13
Shall we close this ticket then?
msg231298 - (view) Author: Craig (craigemery) Date: 2014-11-17 21:30
Surely if there is data loss this *has* to be fixed in 2.x?
I'm seeing this a *lot* using Twisted which, I believe isn't fully 3.x ported yet, so we have to support 2.x for now.
msg231299 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-11-17 21:42
The simple fact of the matter is that socket.makefile() has never worked for non-blocking sockets on 2.x. I doubt Twisted would be so broken as to use it in that context, but if that's the case, you should report a bug to them. If it's your own code, just use the socket directly (and call recv() and friends).

3.x should work fine, in that read() will return a short read (or None if nothing could be read before EAGAIN). Still, more complex calls such as readline() may still give incorrect results.
msg231300 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-11-17 21:42
(closed as won't fix, sorry)
Date User Action Args
2022-04-11 14:56:07adminsetgithub: 40971
2014-11-17 21:42:37pitrousetmessages: + msg231300
2014-11-17 21:42:15pitrousetstatus: languishing -> closed
resolution: wont fix
messages: + msg231299
2014-11-17 21:30:44craigemerysetnosy: + craigemery
messages: + msg231298
2014-02-03 19:53:28BreamoreBoysetnosy: - BreamoreBoy
2013-07-08 16:13:51christian.heimessetstatus: open -> languishing
nosy: + christian.heimes
messages: + msg192666

2011-01-04 01:33:15pitrousetnosy: + pitrou

messages: + msg125275
versions: - Python 3.1, Python 3.2
2010-08-19 16:39:37BreamoreBoysetnosy: + BreamoreBoy

messages: + msg114378
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6
2009-02-14 18:21:49ajaksu2setstage: test needed
type: behavior
versions: + Python 2.6
2004-10-01 16:56:04jnelsoncreate