classification
Title: Select() failure (race condition)
Type: behavior Stage: test needed
Components: Extension Modules, Library (Lib) Versions: Python 3.0, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ajaksu2, amaury.forgeotdarc, gagenellina, rongarret (4)
Priority: normal Keywords

Created on 2007-04-22 20:38 by rongarret, last changed 2009-04-03 17:29 by amaury.forgeotdarc.

Messages (5)
msg31865 - (view) Author: Ron Garret (rongarret) Date: 2007-04-22 20:38
select.select fails on file-like objects created by socket.makefile when data exists in the file buffer but not in the underlying socket.

Here is code to reproduce the bug.  Run it, then point a web browser at port 8080 and observer that select times out indicating that no input is available even though input is still in fact available.

=======

from SocketServer import *
from socket import *
from select import select

class myHandler(StreamRequestHandler):
  def handle(self):
    while 1:
      sl = select([self.rfile],[],[],1)[0]
      print sl
      l = self.rfile.readline()
      if len(l)<3: break
      print l,
      pass
    print>>self.wfile, 'HTTP/1.0 200 OK'
    print>>self.wfile, 'Content-type: text/plain'
    print>>self.wfile
    print>>self.wfile, 'foo'

server = TCPServer(('',8080), myHandler)
server.serve_forever()
msg31866 - (view) Author: Gabriel Genellina (gagenellina) Date: 2007-04-23 04:53
The handler should not do its own select() call. (And even then, select should use the underlying socket, not the wrapping pseudo-file object).
Just removing the select() call works fine.
msg31867 - (view) Author: Ron Garret (rongarret) Date: 2007-04-23 06:00
>Just removing the select() call works fine.

For this simple example yes, not for more complicated cases.  My particular application is a transparent HTTP proxy which needs to be able to serve multiple connections at once and handle keep-alive connections.  (The proxy connects to local server processes through unix sockets, which is why I can't use a regular HTTP proxy.  Send me email if you want more details about why I'm doing this.)  Without writing a full-blown HTTP-aware proxy, select (or something like it) is necessary.

Upon further study, though, I have come to the conclusion that this is not actually a bug (since select is doing what it is advertised to do), just very counter-intituitive behavior.  If select is going to accept file objects as arguments it ought to do the Right Thing with them IMO.
msg84716 - (view) Author: Daniel Diniz (ajaksu2) Date: 2009-03-30 23:25
Cannot verify for trunk.
msg85311 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) Date: 2009-04-03 17:29
This older post 
http://bytes.com/groups/python/786579-python-2-2-1-select
describes a similar problem where select() is used on a buffered file
object (a pipe to another process)

IMO it should be documented that select() does not work so well for
objects which have a fileno(), but do data buffering: select() only sees
the file descriptor and cannot know whether there is data in the buffer.
History
Date User Action Args
2009-04-03 17:29:23amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg85311
2009-03-30 23:25:41ajaksu2setnosy: + ajaksu2
versions: + Python 2.6, Python 3.0, - Python 2.5
messages: + msg84716

components: + Extension Modules
type: behavior
stage: test needed
2007-04-22 20:38:27rongarretcreate