classification
Title: IDLE - readline, isatty, and input broken
Type: behavior Stage: needs patch
Components: IDLE Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Ramchandra Apte, loewis, ned.deily, python-dev, roger.serwy, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2012-07-10 19:28 by roger.serwy, last changed 2012-07-11 09:00 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
stdin_fix.patch roger.serwy, 2012-07-10 19:28 review
idle_stdin_no_rpcfile.patch serhiy.storchaka, 2012-07-10 20:26 review
idle_stdstreams.patch serhiy.storchaka, 2012-07-11 08:51 review
Messages (14)
msg165198 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-07-10 19:28
Per Martin's request in msg165197, this issue has been separated from Issue13532.

The newly introduced _RPCFile object in run.py (422242dbce30) inherits from io.TextIOBase which has "readline" as one of its methods. As a result, IDLE's internal call readline are not handled properly.

The stdin_fix.patch provides a fix for readline and isatty.
msg165202 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-07-10 20:26
There is a simpler solution of this issue.
msg165204 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-07-10 21:20
I like Serhiy's patch. It works for me.
msg165214 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-07-11 01:13
I agree that the _RPCFile wrapping of the stdin proxy should be undone unless and until there is a positive reason for it -- it solves a problem -- and it is better tested. But reversion does not solve pre-existing problems. As noted by Roger, sys.stdin writes, which it should not do. Worse, when directly used, it does not read, which it should do.

>>> sys.stdin
<idlelib.rpc.RPCProxy object at 0x0000000003282470>
>>> dir(sys.stdin)
['_RPCProxy__attributes', '_RPCProxy__getattributes', '_RPCProxy__getmethods', '_RPCProxy__methods', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'encoding', 'oid', 'sockio']
>>> sys.stdin.write('abc')
abc
>>> sys.stdin.read()
Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    sys.stdin.read()
AttributeError: read

versus, in command interpreter (Win7,3.3.0b0)

>>> sys.stdin.read()
abcdefg
^Z
'abcdefg\n'
>>> sys.stdin.write('abc')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
io.UnsupportedOperation: not writable

Same difference for readlines and writelines. I wonder how input works if it does not call sys.stdin.read()

I found this in rpc.py:
# XXX KBK 09Sep03  We need a proper unit test for this module.
# Previously existing test code was removed at Rev 1.27 (r34098).
msg165217 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-07-11 01:44
Including issue15318 where stdin is writable. The proper solution to that issue and this one are likely the same.
msg165218 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-07-11 01:47
> Same difference for readlines and writelines. I wonder how input works if it does not call sys.stdin.read()
>

Eventually IDLE makes a call to PyShell's readline.
msg165222 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-07-11 03:11
Ah, sys.stdin.readline() works. So the object being proxied has readline (and write and writelines) but not read and readlines. I presume the proxy is in the user process and the actual stdin/out/err objects are in the idle process, interacting with the screen and keyboard via tkinter. Do you know what and where it is, so that we could look at it?
msg165225 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-07-11 05:33
PyShell.py's PyShell object has the readline method, at line 1080 in the most recent code. It's meant for use with and without a subprocess. (See also Issue14254)

The IDLEfork project long ago created the subprocess and the RPC plumbing for the subprocess interaction with the *existing* PyShell(OutputWindow) object as stdin/stdout/stderr. This RPC, found in rpc.py, is used in run.py and PyShell's ModifiedInterpreter. The start_subprocess method connects the stdin/stdout/stderr from the IDLE front-end to the subprocess in run.py. The stdout/stderr objects are the PyShell(OutputWindow) object wrapped by PseudoFile. Stdin is not wrapped, but it should be.

I know that this issue deals with stdin specifically, but I hope that the additional stdout/stderr information places the problem into a greater context.
msg165226 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-07-11 06:15
I think the issue is grossly underspecified. The title of the issue is "sys.stdin is broken" which says nothing about the way in which it is broken. Taking Roger's literal description, and his msg165191, the complaint is that input() no longer works, as doesn't readline and isatty.

Since this was what Roger opened the issue for, I declare this to be the issue being dealt with here. I agree that Serhiy's patch is a proper resolution of the issue.

Roger, please refrain from trying to put issues in a broader scope, unless you make an explicit specification what exactly is in scope and what is out of scope. I readily agree that there are many issues with stdin and stdout in IDLE; I don't agree that easily that they all deserve being resolved. I entirely, completely, absolutely disagree that they are related to what you originally submitted as this issue, as I firmly, deeply convinced believe that the fixes to the other issues will be different from the proper fix to this issue. So please, please, please don't mix issues.
msg165227 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-07-11 06:30
New changeset b90482742ee0 by Martin v. Löwis in branch '3.2':
Issue #15319: Revert wrapping of sys.stdin. Patch by Serhiy Storchaka.
http://hg.python.org/cpython/rev/b90482742ee0
msg165229 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-07-11 06:32
New changeset 42424c1f605c by Martin v. Löwis in branch '2.7':
Issue #15319: Revert wrapping of sys.stdin. Patch by Serhiy Storchaka.
http://hg.python.org/cpython/rev/42424c1f605c
msg165236 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-07-11 08:51
> title: IDLE - input() is broken. -> IDLE - sys.stdin is broken.

Well, now, with the modified header, I'm not going to open a separate
issue. Here is a patch that fixes almost all IDLE sys.std* issues.
msg165237 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-07-11 08:53
Serhiy: this issue is closed, so your patch won't be considered.
msg165238 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-07-11 09:00
Sorry, I missed the morning's discussion.
History
Date User Action Args
2012-07-11 09:00:21serhiy.storchakasetmessages: + msg165238
title: IDLE - sys.stdin, sys.stdout, etc are broken -> IDLE - readline, isatty, and input broken
2012-07-11 08:53:51loewissetmessages: + msg165237
2012-07-11 08:51:57serhiy.storchakasetfiles: + idle_stdstreams.patch

messages: + msg165236
title: IDLE - readline, isatty, and input broken -> IDLE - sys.stdin, sys.stdout, etc are broken
2012-07-11 06:35:03loewisunlinkissue15318 superseder
2012-07-11 06:32:52loewissetstatus: open -> closed
resolution: fixed
2012-07-11 06:32:16python-devsetmessages: + msg165229
2012-07-11 06:30:53python-devsetnosy: + python-dev
messages: + msg165227
2012-07-11 06:15:22loewissetmessages: + msg165226
title: IDLE - sys.stdin is broken. -> IDLE - readline, isatty, and input broken
2012-07-11 05:33:42roger.serwysetmessages: + msg165225
2012-07-11 04:34:02Ramchandra Aptesetnosy: + Ramchandra Apte
2012-07-11 03:12:16terry.reedysettitle: IDLE - input() is broken. -> IDLE - sys.stdin is broken.
2012-07-11 03:11:11terry.reedysetmessages: + msg165222
2012-07-11 01:47:25roger.serwysetmessages: + msg165218
2012-07-11 01:44:37roger.serwysetmessages: + msg165217
2012-07-11 01:43:46roger.serwylinkissue15318 superseder
2012-07-11 01:13:51terry.reedysetstage: needs patch
messages: + msg165214
versions: + Python 3.2
2012-07-10 21:20:56roger.serwysetmessages: + msg165204
2012-07-10 20:26:37serhiy.storchakasetfiles: + idle_stdin_no_rpcfile.patch

nosy: + serhiy.storchaka
messages: + msg165202

type: behavior
2012-07-10 19:28:47roger.serwycreate