classification
Title: IDLE's shell returns a multiple-line string to input() or readline() when multiple lines of text are pasted by the user
Type: behavior Stage: resolved
Components: IDLE Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Steven.Barker, kbk, python-dev, roger.serwy, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2013-12-23 22:28 by Steven.Barker, last changed 2013-12-25 12:28 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
idle_readline.patch serhiy.storchaka, 2013-12-24 07:43 review
Messages (5)
msg206879 - (view) Author: Steven Barker (Steven.Barker) * Date: 2013-12-23 22:28
Pasting multiple lines of input and then pressing Enter when IDLE is waiting to read a single line (such as when input() or sys.stdin.readline() have been called) will result is a multi-line string being given as the input, rather than a single line.

This may be most easily understood by looking at an example. Run this code in IDLE (either by typing it in the shell, or from a file with F5):

    s = "X"
    while s:
        s = input()
        print(repr(s))

First, try typing in several lines. Each will be printed separately, with no newlines inside the strings (since input() strips a trailing newline).

    foo
    'foo'
    bar
    'bar'
    baz
    'baz'

Next, copy several lines of text from somewhere. It doesn't matter what the lines' contents are. Here I grabbed a list of Python version numbers, as I was on the download page after grabbing 3.4.0b1 for testing this bug:

    3.1.5
    3.0.1
    2.7.6
    2.6.9
    2.5.6
    2.4.6
    2.3.7
    2.2.3 <all the preceding lines were pasted in one go, followed by enter pressed here>
    '3.1.5\n3.0.1\n2.7.6\n2.6.9\n2.5.6\n2.4.6\n2.3.7\n2.2.3'

This behavior is different than what the Python interpreter does in a regular console shell. When running in cmd.exe on Windows, Python treats a multi-line paste just like typed input:

    3.1.5
    '3.1.5'
    3.0.1
    '3.0.1'
    2.7.6
    '2.7.6'
    2.6.9
    '2.6.9'
    2.5.6
    '2.5.6'
    2.4.6
    '2.4.6'
    2.3.7
    '2.3.7'
    2.2.3 <enter typed here>
    '2.2.3'

I expect the same behavior will be common in other kinds of terminals on other platforms.

This issue makes testing certain kinds of programs very frustrating. If your program needs to read certain text from STDIN, and you want to paste that text in quickly, you need to update your code with special logic just for use in IDLE's console. As an example of the kind of pain you may experience, try copying and pasting a block of text with a blank line into the input loop above. On a regular console session it will exit the loop after the blank line. In IDLE, it will keep running.

I've traced the source of this issue through IDLE's sys.stdin file object and an RPC call, and found it probably is located in the idlelib.PyShell.PyShell.readline method (or the surrounding code). This grabs a string from the Text object in the shell window and returns it to the Python code running in the subprocess.

Probably it should have some extra steps added to check if it got multiple lines. If so, it should split the string on newlines and return just one line of text for each readline call. I'm not sure exactly what should be done with the rest of the lines, but perhaps they could be queued up (or somehow "put back" by moving the markers in the Text object) so later lines would be grabbed by later input requests.

Or alternatively, maybe the event where the multi-line paste arrives should be handled differently, as several single-line input events, rather than a single multiple-line one.
msg206889 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-24 07:43
Here is simple patch.
msg206920 - (view) Author: Roundup Robot (python-dev) Date: 2013-12-25 12:25
New changeset 526dcd51e425 by Serhiy Storchaka in branch '2.7':
Issue #20058: sys.stdin.readline() in IDLE now always returns only one line.
http://hg.python.org/cpython/rev/526dcd51e425

New changeset 8f75d8ddc95b by Serhiy Storchaka in branch '3.3':
Issue #20058: sys.stdin.readline() in IDLE now always returns only one line.
http://hg.python.org/cpython/rev/8f75d8ddc95b
msg206921 - (view) Author: Roundup Robot (python-dev) Date: 2013-12-25 12:27
New changeset 2a4c083f8f6b by Serhiy Storchaka in branch 'default':
Issue #20058: sys.stdin.readline() in IDLE now always returns only one line.
http://hg.python.org/cpython/rev/2a4c083f8f6b
msg206922 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-25 12:28
Fixed. Thank you Steven for your report.
History
Date User Action Args
2013-12-25 12:28:23serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg206922

stage: patch review -> resolved
2013-12-25 12:27:22python-devsetmessages: + msg206921
2013-12-25 12:25:33python-devsetnosy: + python-dev
messages: + msg206920
2013-12-25 11:41:06serhiy.storchakasetassignee: serhiy.storchaka
2013-12-24 07:43:50serhiy.storchakasetfiles: + idle_readline.patch
keywords: + patch
messages: + msg206889

stage: patch review
2013-12-24 07:05:05serhiy.storchakasetnosy: + terry.reedy, kbk, roger.serwy, serhiy.storchaka
2013-12-23 22:28:33Steven.Barkercreate