>>> a = subprocess.Popen(['cat', '/path/to/text.ini'], stdout=subprocess.PIPE, universal_newlines=True)
>>> b = configparser.ConfigParser()
>>> b.read_file(a.stdout)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/hostname/sig/local/lib/python3.2/configparser.py", line 708, in read_file
    self._read(f, source)
  File "/hostname/sig/local/lib/python3.2/configparser.py", line 994, in _read
    for lineno, line in enumerate(fp, start=1):
AttributeError: '_io.FileIO' object has no attribute 'read1'

Also this fails without universal_readlines, which is not so bad except that the name 'read_file' doesn't exactly indicate this.

I found one mildly related bug: http://bugs.python.org/issue11670

This isn't a problem with ConfigParser, it's a problem with the text file object returned by Subprocess. You can reproduce the bug by trying to iterate over the TextIOWrapper returned by subprocess.

(It is true, however, that the ConfigParser docs should be specific that it isn't "any file", but "any text file". If you would like to open a separate issue about that, that would be great).

The "universal_newlines" feature is rather poorly named in Python 3.x, since it does much more than that (the resulting files yield and expect unicode strings rather than bytes objects).

The problem is that io.TextIOWrapper needs a buffered I/O object, but bufsize is 0 by default in subprocess.Popen(). In the meantime, you can workaround it using a non-zero bufsize in the Popen() constructor. Of course, this has the side-effect of forcing you to call flush() on the stdin stream (if you are using it).

So this is a doc bug in subprocess? Explaining this clearly doesn't sound like much fun...

So, as the title indicates, I think we should make TextIOWrapper work with raw IO objects. The reason is so that write() can behave in a totally unbuffered way, which is necessary for Popen to behave appropriately.

Hmm, one problem is that the C TextIOWrapper buffers writes. We would need an additional constructor parameter to prevent that :/

Another possibility is to provide read1() on RawIO, as a synonym of read().

Here is a first patch allowing TextIOWrapper to work with raw IO objects, and adding a "write_through" flag.

"write_through" is not used in _pyio.py, is it expected?

"write_through" is not used in _pyio.py, is it expected?

Yup, because it is actually always write-through (writes are not
cached). The argument is only there so that the signature is the same as
the C version.

Looks good, then.

This additional patch improves universal_newlines support in subprocess (still broken with the select- and poll-based loops in communicate()).

New changeset 9144014028f3 by Antoine Pitrou in branch '3.2':
Issue bpo-12591: Allow io.TextIOWrapper to work with raw IO objects (without
http://hg.python.org/cpython/rev/9144014028f3

New changeset c3b47cdea0d1 by Antoine Pitrou in branch 'default':
Issue bpo-12591: Allow io.TextIOWrapper to work with raw IO objects (without
http://hg.python.org/cpython/rev/c3b47cdea0d1

New changeset 5cc536fbd7c1 by Antoine Pitrou in branch '3.2':
Issue bpo-12591: Improve support of "universal newlines" in the subprocess
http://hg.python.org/cpython/rev/5cc536fbd7c1

New changeset b616396fa170 by Antoine Pitrou in branch 'default':
Issue bpo-12591: Improve support of "universal newlines" in the subprocess
http://hg.python.org/cpython/rev/b616396fa170

Ok, this should be fixed now. universal newlines support is still broken with communicate(), I've opened bpo-12623 for that.