Author yann
Recipients yann
Date 2020-07-06.19:54:01
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1594065242.21.0.841046846995.issue41222@roundup.psfhosted.org>
In-reply-to
Content
On a POpen object created with bufsize=0, stdout.readline() does a buffered reading with python3, whereas in 2.7 it did char-by-char reading.  See attached example.

As a result, a poll including the stdout object suffers a behaviour change when stdout is ready for writing and there is more than one line of data available.  In both cases we get notified by poll() that data is available on the fd and we can stdout.readline() and get back to our polling loop.  Then:

* with python2 poll() then returns immediately and stdout.readline() will then return the next line

* with python3 poll() now blocks

Running the attached example under strace reveals the underlying difference:

 write(4, "go\n", 3)                     = 3
 poll([{fd=5, events=POLLIN|POLLERR|POLLHUP}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "\n", 1)                        = 1
-fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x2), ...}) = 0
+read(5, "xxxxxxxxxxxx\nyyyyyyyyyyyyyyy\naaa"..., 8192) = 74
 write(1, ">xxxxxxxxxxxx\n", 14)         = 14


We can see a buffered read, which explains the behaviour difference.

Changing to bufsize=1, strace does not show a difference here.

This is especially troubling, as the first note in https://docs.python.org/3/library/io.html#class-hierarchy mentions that even in buffered mode there is an unoptimized readline() implementation.
History
Date User Action Args
2020-07-06 19:54:02yannsetrecipients: + yann
2020-07-06 19:54:02yannsetmessageid: <1594065242.21.0.841046846995.issue41222@roundup.psfhosted.org>
2020-07-06 19:54:02yannlinkissue41222 messages
2020-07-06 19:54:02yanncreate