classification
Title: setblocking() method on file objects
Type: enhancement Stage: needs patch
Components: IO, Library (Lib) Versions: Python 3.2
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: abo, ajaksu2, brian.curtin, giampaolo.rodola, haypo, pitrou, tim.golden
Priority: normal Keywords: patch

Created on 2004-05-07 05:26 by abo, last changed 2010-10-27 12:20 by haypo.

Files
File name Uploaded Description Edit
io_blocking.patch haypo, 2010-10-27 12:19 review
test_process.py haypo, 2010-10-27 12:20
Messages (6)
msg60484 - (view) Author: Donovan Baarda (abo) Date: 2004-05-07 05:26
Currently file object's write() method returns nothing.
It would be very nice if it returned the number of
bytes written. It would also be nice if file objects
had a setblocking() method, similar to socket objects.

Under various situations, such as communicating with
popen'ed processes, write() can block. To avoid this,
you must set the file to non-blocking mode, or do
something like fork a thread to service the process so
that it can "unblock" the write.

Currently, setting a file to unblocking mode requires
using fnctl on the file objects fileno(). It would be
nice if you could directly modify this file behaviour
using a setblocking() method.

When you do set a file to non-blocking mode, the file
object's write() method will raise an exception if not
all the data could be written. This is useless, as
there is no indication how much data was successfuly
written before the blocking happened, so you have no
idea how much data remains to be written when the file
becomes unblocked.

Even using things like select() loops are not reliable,
as write() on an unblocked file can block part way
through even a small data buffer. 

The only sure way to avoid a file object's write() from
blocking or raising an unhelpful exception is to write
a single byte at a time. This means you must use
os.write() instead. The os.write() method returns the
number of bytes written, which tells how many bytes of
data remain to be written next time when the file
becomes unblocked.

The existing read() method behaves the same as the
os.read() method and can be used in non-blocking mode.
If the file write() method behaved the same as the
os.write() method, returning the number of bytes
written instead of raising an exception, then it could
also be used in non-blocking mode. 
msg82049 - (view) Author: Daniel Diniz (ajaksu2) Date: 2009-02-14 13:54
Implemented in 3.0, 2.x probably won't get this. Will close if nobody
opposes.
msg82473 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-19 13:53
setblocking() doesn't exist in py3k either, so reopening. I agree it
would be useful to set files as non-blocking in a portableway.
msg113427 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) (Python committer) Date: 2010-08-09 16:51
Any clue on where to start to do this?
What about Windows where fcntl is not available?
msg113431 - (view) Author: Tim Golden (tim.golden) (Python committer) Date: 2010-08-09 17:45
There are at least two ways to do non-blocking file IO on Windows:

Overlapped I/O
I/O Completion ports

Don't know what's best here, but happy to see what might be achieved
if it was thought worth pursuing.
msg119704 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-10-27 12:19
Prototype to test nonblocking file objet:
 - add getblocking() and setblocking() methods to _io._FileIO and all _pyio classes
 - fileio_setblocking() is implemented using fcntl(fd, F_SETFL, flags | O_NONBLOCK) (POSIX only?)
 - BufferedReader.read() uses read1() if the file is blocking

Use test_process.py to test it: this script runs a Python interpreter in a subprocess. It uses select() to check if there is data or not. Eg. type '1+1\n' and then 'exit()\n'. Set PYIO_HAVE_BLOCKING constant to False (in test_process.py) to test the script without io_blocking.patch.

I'm not sure that select() is required, but it doesn't work without it (read() blocks).
History
Date User Action Args
2010-10-27 12:20:08hayposetfiles: + test_process.py
2010-10-27 12:19:52hayposetfiles: + io_blocking.patch

nosy: + haypo
messages: + msg119704

keywords: + patch
2010-08-09 18:18:31brian.curtinsetnosy: + brian.curtin
2010-08-09 17:45:41tim.goldensetnosy: + tim.golden
messages: + msg113431
2010-08-09 16:52:21giampaolo.rodolasetnosy: abo, pitrou, giampaolo.rodola, ajaksu2
components: + IO
2010-08-09 16:51:39giampaolo.rodolasetnosy: + giampaolo.rodola
messages: + msg113427
2010-08-09 16:38:49pitrousetstage: test needed -> needs patch
2010-08-09 03:22:30terry.reedysetstage: test needed
versions: + Python 3.2, - Python 3.1
2009-02-19 13:54:08pitrousettitle: file write() method and non-blocking mode. -> setblocking() method on file objects
2009-02-19 13:53:36pitrousetstatus: pending -> open
priority: low -> normal
messages: + msg82473
nosy: + pitrou
versions: + Python 3.1
2009-02-18 01:51:46ajaksu2setstatus: open -> pending
priority: normal -> low
2009-02-14 13:54:48ajaksu2setnosy: + ajaksu2
messages: + msg82049
2004-05-07 05:26:57abocreate