classification
Title: Document select() failure with buffered file
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: ajaksu2, amaury.forgeotdarc, docs@python, ggenellina, rongarret
Priority: normal Keywords: easy

Created on 2007-04-22 20:38 by rongarret, last changed 2021-12-05 16:59 by iritkatriel.

Messages (6)
msg31865 - (view) Author: Ron Garret (rongarret) Date: 2007-04-22 20:38
select.select fails on file-like objects created by socket.makefile when data exists in the file buffer but not in the underlying socket.

Here is code to reproduce the bug.  Run it, then point a web browser at port 8080 and observer that select times out indicating that no input is available even though input is still in fact available.

=======

from SocketServer import *
from socket import *
from select import select

class myHandler(StreamRequestHandler):
  def handle(self):
    while 1:
      sl = select([self.rfile],[],[],1)[0]
      print sl
      l = self.rfile.readline()
      if len(l)<3: break
      print l,
      pass
    print>>self.wfile, 'HTTP/1.0 200 OK'
    print>>self.wfile, 'Content-type: text/plain'
    print>>self.wfile
    print>>self.wfile, 'foo'

server = TCPServer(('',8080), myHandler)
server.serve_forever()
msg31866 - (view) Author: Gabriel Genellina (ggenellina) Date: 2007-04-23 04:53
The handler should not do its own select() call. (And even then, select should use the underlying socket, not the wrapping pseudo-file object).
Just removing the select() call works fine.
msg31867 - (view) Author: Ron Garret (rongarret) Date: 2007-04-23 06:00
>Just removing the select() call works fine.

For this simple example yes, not for more complicated cases.  My particular application is a transparent HTTP proxy which needs to be able to serve multiple connections at once and handle keep-alive connections.  (The proxy connects to local server processes through unix sockets, which is why I can't use a regular HTTP proxy.  Send me email if you want more details about why I'm doing this.)  Without writing a full-blown HTTP-aware proxy, select (or something like it) is necessary.

Upon further study, though, I have come to the conclusion that this is not actually a bug (since select is doing what it is advertised to do), just very counter-intituitive behavior.  If select is going to accept file objects as arguments it ought to do the Right Thing with them IMO.
msg84716 - (view) Author: Daniel Diniz (ajaksu2) * (Python triager) Date: 2009-03-30 23:25
Cannot verify for trunk.
msg85311 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-04-03 17:29
This older post 
http://bytes.com/groups/python/786579-python-2-2-1-select
describes a similar problem where select() is used on a buffered file
object (a pipe to another process)

IMO it should be documented that select() does not work so well for
objects which have a fileno(), but do data buffering: select() only sees
the file descriptor and cannot know whether there is data in the buffer.
msg116665 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-09-17 13:58
I've changed things in reply to msg85311, feel free to alter things again if you disagree.
History
Date User Action Args
2021-12-05 16:59:15iritkatrielsetkeywords: + easy
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 2.7, Python 3.5, Python 3.6
2016-05-14 11:51:49martin.pantersettitle: Select() failure (race condition) -> Document select() failure with buffered file
versions: + Python 3.5, Python 3.6, - Python 3.1, Python 3.2
2010-12-26 01:46:24eric.araujosetnosy: - BreamoreBoy

stage: test needed -> needs patch
2010-09-17 13:58:20BreamoreBoysetversions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 3.0
nosy: + BreamoreBoy, docs@python

messages: + msg116665

assignee: docs@python
components: + Documentation, - Extension Modules, Library (Lib)
2009-04-03 17:29:23amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg85311
2009-03-30 23:25:41ajaksu2settype: behavior
components: + Extension Modules
versions: + Python 2.6, Python 3.0, - Python 2.5
nosy: + ajaksu2

messages: + msg84716
stage: test needed
2007-04-22 20:38:27rongarretcreate