Title: Bracketed paste mode for REPL: don't execute pasted command before ENTER is pressed explicitly
Type: enhancement Stage:
Components: Interpreter Core Versions: Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Marco Sulla, asmeurer, eric.smith, eryksun, steven.daprano, terry.reedy
Priority: normal Keywords:

Created on 2020-03-01 23:30 by Marco Sulla, last changed 2021-02-17 22:01 by vstinner.

Messages (14)
msg363109 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-03-01 23:30
I suggest to add an implementation of bracketed paste mode in the REPL.

Currently if you, for example, copy & paste a piece of Python code to see if it works, if the code have a blank line without indentation and the previous and next line are indented, REPL raises an error.
If you create a .py, paste the same code and run it with the python interpreter, no error is raised, since the syntax is legit.

Bracketed paste mode is implemented in many text editors, as vi.
msg363139 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-03-02 08:24
Is this even possible in a plain text console (or plain text console + GNU readline-equivalent)?

> Bracketed paste mode is implemented in many text editors, as vi.

Sure, but the REPL is not a text editor, and knows very little about the context it's being run in.
msg363201 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-03-02 19:35
> Is this even possible in a plain text console?

Yes. See Jupyter Console (aka IPython).
msg363258 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-03-03 13:37
See also the discussion in #38747 (also opened by Marco), which was closed as rejected. This issue is close to a duplicate of that one.
msg363295 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-03-03 19:21
Please read the message of Terry J. Reed: 

I quote the relevant part below

> Skipping the rest of your post, I will just restate why I closed this
> issue.
> 1. It introduces too many features not directly related.  The existing
> unix-only completions uses two modules.  I suspect some of the other
> features would also need new modules.  (But Marco, please don't rush 
> to immediately open 8 new issues.)
msg363301 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-03-03 21:21
I suggest closing this issue. If the REPL is going to be improved, it won't be done one-feature-at-a-time on the bug tracker. It will need to have a holistic design that takes into account desired features and the architecture of the systems that it needs to interface with. Windows and Linux may require very different approaches. For such a design, the bug tracker is a poor choice of venue for the discussion. I suggest starting on python-ideas and maybe python-dev until a design and set of features is decided on, and only at that point open issue(s) on the bug tracker.

That said, I'm not sure it's worth complicating CPython itself with such changes, when IPython exists. But that can be decided on python-ideas and then on python-dev.
msg363315 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-03-03 22:35
Excuse me, but my original "holistic" proposal was rejected and it was suggested to me to propose only relevant changes, and one for issue. Now you say exactly the contrary. I feel a bit confused.

PS: yes, I can, and I use, IPython. But IMHO IPython does too much things and its design is not very pythonic. Bracketed paste mode is a good feature, and I think REPL will be much more useful if it implements it.

On the contrary, if you don't think IPython is good, pythonic and essential, I suppose there's no problem to substitute REPL with IPython in CPython core itself.
msg363316 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-03-03 23:06
Reasonable people can disagree. If someone else thinks this can be done piecemeal, then I'm not stopping anyone. I just think that's a mistake and will lead to much frustration.
msg363328 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-03-04 05:44
Adding functions to builtins (2 of Marco's 8 #38747 ideas) should be separate from patching the repl code.  I agree that the latter should not be done piecemeal.  I previous wrote that we need a pydev REPL enhancement policy discussion *first* and explicitly asked Marco "please don't rush to immediately open 8 new issues."  

I do think that if there were an *approved* set of new features, several patches would be best, but that is moot at present.

IDLE already has this feature, and most of the other enhancements Marco wants. (msg363325) The key for this one is that tcl/tk does not trigger <Key> events when inserting pasted code, so only a keyed newline triggers IDLE's user code processing.  But other IDLE features are much harder, and I suspect that this would also be in the Windows console.

In msg356327, Eric Sun, one of our best Windows experts, outlined the major difficulties of changing things on Windows' console.  He only said something half-way encouraging about autoindent but otherwise thought that a working stdlib Windows readline extension would be needed.
msg363336 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-03-04 08:01
IMHO such a feature is useful for sysops that does not have a graphical interface, as Debian without an X. That's why vi is (unluckily) very popular also in 2020. IDLE can't be used in this cases.

Windows users can't remotely login without a GUI, so the feature for such platforms can be not implemented, since there's builtin solutions (IDLE)
msg369299 - (view) Author: Aaron Meurer (asmeurer) Date: 2020-05-18 22:13
Related issue
msg386991 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-02-15 09:49
I marked bpo-32019 as a duplicate of this issue.
msg386992 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-02-15 09:58
See also bpo-42819 "readline 8.1 bracketed paste".
msg387189 - (view) Author: Aaron Meurer (asmeurer) Date: 2021-02-17 21:16
To reiterate some points I made in the closed issues and

A simple "fix" would be to emulate the non-bracketed paste buffering. That is, accept the input using bracketed paste, but split it line by line and send that to the REPL. That would achieve some of the benefits of bracketed paste (faster pasting), without having to change how the REPL works.

For actually allowing multiline input in the REPL, one issue I see is that the so-called "single" compile mode is fundamentally designed around single line evaluation. To support proper multiline evaluation, it would need to break from this model (which in my opinion is over engineered).

In one of my personal projects, I use a function along the lines of 

import ast

def eval_exec(code, g=None, l=None, *, filename="<stdin>", noresult=None):
    if g is None:
        g = globals()
    if l is None:
        l = g
    p = ast.parse(code)
    expr = None
    res = noresult
    if p.body and isinstance(p.body[-1], ast.Expr):
        expr = p.body.pop()
    code = compile(p, filename, 'exec')
    exec(code, g, l)
    if expr:
        code = compile(ast.Expression(expr.value), filename, 'eval')
        res = eval(code, g, l)

    return res

This function automatically execs the code, but if the last part of it is an expression, it returns it (note that this is much more useful than simply printing it). Otherwise it returns a noresult marker (None by default).

I think this sort of functionality in general would be useful in the standard library (much more useful than compile('single')), but even ignoring whether it should be a public function, this is the sort of thing that is needed for "proper" multiline execution in a REPL. Terry mentioned that idle supports multiline already. But I tried pasting

a = 1

into idle (Python 3.9), and I get the same "SyntaxError: multiple statements found while compiling a single statement" error, suggesting it still has the same fundamental limitation. 

Also, if it wasn't clear, I should note that this is independent of pasting. You can already write

def func():
    return 1

manually in the interpreter or IDLE and it will give a syntax error.
Date User Action Args
2021-02-17 22:01:34vstinnersetnosy: - vstinner
2021-02-17 21:16:40asmeurersetmessages: + msg387189
2021-02-15 17:51:45mark.dickinsonsetnosy: - mark.dickinson
2021-02-15 09:58:48vstinnersetmessages: + msg386992
2021-02-15 09:55:23vstinnersettitle: Bracketed paste mode for REPL -> Bracketed paste mode for REPL: don't execute pasted command before ENTER is pressed explicitly
2021-02-15 09:49:20vstinnersetnosy: + vstinner
messages: + msg386991
2021-02-15 09:49:01vstinnerlinkissue32019 superseder
2020-05-18 22:13:59asmeurersetnosy: + asmeurer
messages: + msg369299
2020-03-04 08:01:58Marco Sullasetmessages: + msg363336
2020-03-04 05:44:51terry.reedysetmessages: + msg363328
2020-03-03 23:06:02eric.smithsetmessages: + msg363316
2020-03-03 22:35:46Marco Sullasetnosy: + terry.reedy, steven.daprano, eryksun
messages: + msg363315
2020-03-03 21:21:56eric.smithsetnosy: + eric.smith
messages: + msg363301
2020-03-03 19:21:56Marco Sullasetmessages: + msg363295
2020-03-03 13:37:24mark.dickinsonsetmessages: + msg363258
2020-03-02 19:35:06Marco Sullasetmessages: + msg363201
2020-03-02 08:24:42mark.dickinsonsetnosy: + mark.dickinson
messages: + msg363139
2020-03-01 23:30:35Marco Sullacreate