classification
Title: sys.stdin.read() doesn't return after first EOF on Windows
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: anacrolix, cassava, famart, ggenellina, haypo, martin.panter, ned.deily, ocean-city, pitrou, r_mosaic
Priority: normal Keywords:

Created on 2009-03-18 09:20 by r_mosaic, last changed 2016-03-14 21:52 by berker.peksag. This issue is now closed.

Messages (19)
msg83736 - (view) Author: Fan Decheng (r_mosaic) Date: 2009-03-18 09:20
Platform: Windows Vista x64
Python version: 3.0.1 x64

When I use sys.stdin.readlines(), it correctly ends when I enter ^Z 
(ctrl-Z) on the console (this is the EOF on the console). However when 
I call sys.stdin.read(), it doesn't end when ^Z is entered. It ends 
when I enter the second ^Z.

Repro steps:
1. Open python.
2. Type:
import sys
sys.stdin.read()
3. Type:
Hello
^Z
4. Note the above ^Z should be followed by a Return.
Result:
The function call doesn't end. If I enter another ^Z, it ends.
Expected result:
The function call ends.
msg83742 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-18 10:51
(cannot reproduce under Linux)
msg83755 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-03-18 16:47
I can reproduce this on coLinux(debian). But coLinux is running on
windows, so its console behavior may not be same as native linux.
(You need to use Ctrl+D on coLinux instead of Ctrl+Z)
msg83756 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-03-18 16:55
With following patch for investigation, (release30-maint)

Index: Lib/io.py
===================================================================
--- Lib/io.py	(revision 70450)
+++ Lib/io.py	(working copy)
@@ -57,6 +57,7 @@
 
 import os
 import abc
+import sys
 import codecs
 import _fileio
 # Import _thread instead of threading to reduce startup cost
@@ -931,6 +932,7 @@
             while True:
                 # Read until EOF or until read() would block.
                 chunk = self.raw.read()
+                print("============>", repr(chunk), file=sys.stderr)
                 if chunk in empty_values:
                     nodata_val = chunk
                     break

///////////////////////

I got this result.

>>> sys.stdin.read()
abc
^Z
============> b'abc\n'
^Z
============> b''
'abc\n'

To get empty chunk, we need to hit CTRL-Z twice.
msg84145 - (view) Author: Fan Decheng (r_mosaic) Date: 2009-03-25 07:33
Perhaps using just one read() is enough? I mean perhaps no loop is 
necessary?
msg84166 - (view) Author: (cassava) Date: 2009-03-25 20:49
This is happening on Arch Linux as well:

Python 3.0.1 (r301:69556, Feb 22 2009, 02:43:30) 
[GCC 4.3.3] on linux2

I tried python2.6 and it ends with one Ctrl+D
I tried python3.0.1 and it needs two Ctrl+D before it ends
msg91154 - (view) Author: Martin (famart) Date: 2009-07-31 21:38
Looks like python needs eof() or something for file objects, just like
any other languages.

Since read() is using the system call, that's the right behavior: read()
blocks until EOF, and returns whatever was buffered. EOF character is
consumed, but since it's a stdin, it is never closed. The next read()
will again wait for normal input.

The 2nd EOF mark without anything in-between will return an empty string.
msg91296 - (view) Author: Gabriel Genellina (ggenellina) Date: 2009-08-05 02:13
This is a duplicate of #1633941
msg139994 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-07 21:08
I am still able to reproduce the problem with Python 3.2.1RC1 (64 bits) on Windows Seven, but not on Python 3.3 (alpha) compiled myself (using Visual C++ Express 2008).

I don't know if something changed in Python 3.3, or it is related to how I compiled Python? I don't think that it is related to issue #12016 because Python 3.2.1RC1 contains the fix for #12016.
msg139995 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-07 21:37
> I don't know if something changed in Python 3.3, or ...

Yes, something changed in Python 3.3. I fixed this issue "by mistake" :-) The fix is the following commit:

New changeset 3c7792ec4547 by Victor Stinner in branch 'default':
Issue #12175: BufferedReader.read(-1) now calls raw.readall() if available.
http://hg.python.org/cpython/rev/3c7792ec4547

It is a bug in BufferedReader, a bug or an unexpected behaviour. Before this commit (e.g. in Python 3.2.0), BufferedReader.read(-1) calls raw.read() two times for this issue: the first call returns an non empty string (the string until the first ^Z), the second call returns an empty string (the string until the second ^Z). BufferedReader.read(-1) loop ends with raw.read() returns an empty string.

You can workaround this issue by calling sys.stdin.buffer.raw.readall() or sys.stdin.buffer.raw.read(-1).

I chose to not port my BufferedRead.read(-1) fix (call raw.readall() if available) in Python 3.2 because it changes the behaviour, and I don't want to do that in a minor release.

Is anyone in favor of changing the behaviour of BufferedRead.read(-1) in a minor release (next 2.7.3 and 3.2.2)? Or can I close the issue (the bug is already fixed in Python 3.3)?
msg140322 - (view) Author: Matt Joiner (anacrolix) Date: 2011-07-14 04:42
I get this on Linux with ^D
msg140326 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-14 10:44
> I get this on Linux with ^D

With which Python version? Did you try Python 3.3 (development version)?
msg140561 - (view) Author: Matt Joiner (anacrolix) Date: 2011-07-18 07:10
Feel like a total noob: Where do I get the latest source? I can't find any pre-release tarballs for 3.3, and the suggested py3k checkout doesn't work: $ hg clone http://hg.python.org/cpython#py3k py3k
abort: unknown revision 'py3k'!
msg140562 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-07-18 07:14
See the developer's guide: http://docs.python.org/devguide/setup.html#getting-the-source-code

   hg clone http://hg.python.org/cpython directory_name
msg140563 - (view) Author: Matt Joiner (anacrolix) Date: 2011-07-18 07:17
This version is fixed for me:

$ ./python
Python 3.3.0a0 (default:7520f1bf0a81, Jul 18 2011, 17:12:12)
[GCC 4.1.2 20070115 (SUSE Linux)] on linux2
msg140564 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-18 07:38
@pitrou: Antoine, do you think that the following commit should be backported from 3.3 to 3.2?

New changeset 3c7792ec4547 by Victor Stinner in branch 'default':
Issue #12175: BufferedReader.read(-1) now calls raw.readall() if available.
http://hg.python.org/cpython/rev/3c7792ec4547

It changes BufferedReader.read() behaviour a *little* bit. Only a little because FileIO.read(-1) calls FileIO.readall() internally for example.
msg140572 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-07-18 11:32
> Antoine, do you think that the following commit should be backported
> from 3.3 to 3.2?

No, I don't think so.
msg140573 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-18 12:17
> No, I don't think so.

The issue is already fixed in 3.3, so you agree to not fix it in Python 3.2?
msg261723 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-03-14 04:44
Since this was apparenly only a bug in 3.2, can we close it as being out of date?
History
Date User Action Args
2016-03-14 21:52:23berker.peksagsetstatus: open -> closed
resolution: out of date
stage: resolved
2016-03-14 04:44:15martin.pantersetnosy: + martin.panter
messages: + msg261723
2011-07-18 12:17:49hayposetmessages: + msg140573
2011-07-18 11:32:42pitrousetmessages: + msg140572
versions: - Python 3.0
2011-07-18 07:38:47hayposetnosy: ggenellina, pitrou, haypo, ocean-city, r_mosaic, ned.deily, cassava, famart, anacrolix
messages: + msg140564
2011-07-18 07:17:53anacrolixsetmessages: + msg140563
versions: + Python 3.2
2011-07-18 07:14:16ned.deilysetnosy: + ned.deily
messages: + msg140562
2011-07-18 07:10:41anacrolixsetmessages: + msg140561
2011-07-14 10:44:30hayposetmessages: + msg140326
2011-07-14 04:42:12anacrolixsetnosy: + anacrolix
messages: + msg140322
2011-07-07 21:37:33hayposetmessages: + msg139995
2011-07-07 21:08:36hayposetmessages: + msg139994
2011-07-07 10:44:56hayposetnosy: + haypo
2010-09-21 15:03:25ocean-citysetmessages: - msg117058
2010-09-21 15:02:05ocean-citysetmessages: + msg117058
2009-08-05 02:13:50ggenellinasetnosy: + ggenellina
messages: + msg91296
2009-07-31 21:38:54famartsetnosy: + famart
messages: + msg91154
2009-03-25 20:49:24cassavasetnosy: + cassava
messages: + msg84166
2009-03-25 07:33:01r_mosaicsetmessages: + msg84145
2009-03-18 16:55:21ocean-citysetmessages: + msg83756
2009-03-18 16:47:12ocean-citysetnosy: + ocean-city
messages: + msg83755
2009-03-18 10:51:57pitrousetnosy: + pitrou
messages: + msg83742
2009-03-18 09:20:28r_mosaiccreate