New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
readline 8.1 enables the bracketed paste mode by default #86985
Comments
Readline 8.1 enables bracketed paste by default. Package managers like Homebrew for macOS and distributions like Arch Linux which use the latest version of readline (released December 2020) now have new behavior when pasting multiline strings into the python REPL. Disabling bracketed paste on 8.1 reverts to the expected behavior, and enabling bracketed paste on 8.0 also reproduces the behavior. Further information in Homebrew/homebrew-core#68193 Example with bracketed paste on: $ pbcopy < in
$ /usr/local/Cellar/python\@3.9/3.9.1_3/bin/python3
Python 3.9.1 (default, Dec 28 2020, 11:22:14)
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+2
3+4
5+6 File "<stdin>", line 1
SyntaxError: multiple statements found while compiling a single statement
Example with it off:
$ /usr/local/bin/python3
Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec 7 2020, 12:10:52)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+2
3
>>> 3+4
7
>>> 5+6
11
>>> |
This bug is present in all versions since the 3.6 branch at least, but was only triggered when users' readline settings explicitly enabled bracketed-paste. Previous reports: https://bugs.python.org/issue32019 and https://bugs.python.org/issue39820 Longer discussion (on a more extensive feature request) at https://bugs.python.org/issue38747 The new factor is that readline 8.1 now enables bracketed-paste by default. So the broken behaviour is seen for Python built with the latest readline, with no specific setting. Python should probably in the long term be improved to handle bracketed-paste, which will make a nicer user experience. But in the short term, it should communication with readline to disable bracketed-paste at runtime, and restore the previous behaviour. |
The linked PR disables bracketed paste regardless of if the user has it as a configuration option or if Python was configured with a version of readline which defaults to on for bracketed paste. Is this a viable approach? |
Bumping this issue because it's a bug affecting all users who build or use a Python built with the current version of readline, to include all macOS HomeBrew and Arch Linux users who use the python from the respective package managers. |
This also affects Fedora 34+ |
Can you try to find why it changed the default? Why should Python change the default readline default behavior? |
I didn't know "bracketed paste mode". It seems like a protocol between a terminal and the user to add markers before/after a pasted text. """
I understand that this mode prevents to run arbitrary command when pasting blindly an untrusted command copied from a webpage, like this example: It can be enabled explicitly by adding "set enable-bracketed-paste" on to your ~/.inputrc configuration file. |
On my Fedora 33 (readline-8.0-5.fc33.x86_64), I enabled the bracketed paste mode by adding "set enable-bracketed-paste" on to my ~/.inputrc config file. When I test https://thejh.net/misc/website-terminal-copy-paste : the evil command is no long executed. If I copy/paste "1+1" and "2+2" commands (two lines pasted at once) manually in the Python REPL, they are no longer executed immediately, Python waits for me pressing ENTER to execute, *as expected*: ---- $ ./python
Python 3.10.0a5+ (heads/master:fcbe0cb04d, Feb 15 2021, 10:30:10)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+1
2+2 But using "./python -i < commands", each line is executed immediately and Python exits after displaying the results: --- $ cat commands
1+1
2+2 $ ./python -i < commands
Python 3.10.0a5+ (heads/master:fcbe0cb04d, Feb 15 2021, 10:30:10)
>>> 2
>>> 4
>>> If I disable again bracketed mode (ex: remove ~/.inputrc file in my case), when I copy/paste manually the two commands at once, they are executed again, as expected: $ ./python
>>> 1+1
2
>>> 2+2
4 The bracketed paste mode can be enabled explicitly in Python with this patch to simulate readline 8.1 default behavior: diff --git a/Modules/readline.c b/Modules/readline.c
index c900e07954..ce8662c000 100644
--- a/Modules/readline.c
+++ b/Modules/readline.c
@@ -217,6 +217,7 @@ readline_read_init_file_impl(PyObject *module, PyObject *filename_obj)
errno = rl_read_init_file(NULL);
if (errno)
return PyErr_SetFromErrno(PyExc_OSError);
+ rl_variable_bind ("enable-bracketed-paste", "on");
Py_RETURN_NONE;
}
All of these behavior look consistent, correct and expected. Note: When I redirect Python output into a file, I cannot see "\e[?2004h" (likely because stdout is not a TTY if it's redirected into a file), so I'm not sure how to check if this escape sequence is injected in Python stdout or not. |
Oh, I forgot to mention that I ran my tests in Gnome Terminal 3.38.1-2.fc33. |
Related issue: test_pdb_interaction_doctest test of pytest fails on Fedora Rawhide because of a "\x1b[?2004h" string:
It fails with readline 8.1 which enables bracketed mode by default, but it fails with any readline version if ~/.inputrc contains "set enable-bracketed-paste on". |
https://lists.gnu.org/archive/html/bug-readline/2020-11/msg00010.html Setting TERM env var to dumb gives me a surprising behavior when I paste "1+1\n2+2" in Python REPL: $ TERM=dumb python3
Python 3.9.1 (default, Jan 20 2021, 00:00:00)
>>> ^J1+1^J2+2^J If I press ENTER, I get: >>> ^J1+1^J2+2^J
File "<stdin>", line 1
1+1
2+2
SyntaxError: multiple statements found while compiling a single statement
Note: if I redirect Python output into a file ("python3 > output"), I get a different behavior since stdout is no longer a TTY. |
There are different things:
For all these reasons, it sounds reasonable to disable the readline bracketed paste mode by default in Python, even if it's enabled explicitly in ~/.inputrc. If an user opts in for the bracketed paste mode, it is more likely to prevent running malicious commands in a shell, rather than not executing immediately commands executed in Python. Once the bpo-39820 will be fixed, we can reconsider to leave the bracketed paste mode default unchanged (no longer disable it explicitly). |
Instead of enabling it by default, why not just keep it but emulate the old behavior by splitting and buffering the input lines? That way you still get some of the benefits of bracketed paste, i.e., faster pasting, but without the hard work of fixing the REPL to actually support native multiline editing + execing multiline statements (the broken "simple" design). |
Thanks Dustin for the bug report *and* the fix! I close the issue. For people who want to support bracketed paste mode in Python, please disuss it in bpo-39820. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: