The following code works on python2 (tested with 2.7.6), but breaks readline for python3 (tested with 3.4.3):

>>> import sys, readline
>>> sys.stdin = open('/dev/tty', 'r')
>>> input("> ")  # press "7<Enter>"
>>> input("> ")  # press "up" key

Re-opening /dev/tty as stdin is a very useful technique. For example, one could debug a script that takes a file as stdin by re-opening the stdin as /dev/tty and using pdb.set_trace().

See bug report for python2 in 2002 here: https://bugs.python.org/issue512981

I think the difference between Python 2 and 3 here is that Python 2’s file objects, including sys.stdin, wrap C library FILE objects, which is supported by the Readline library. However Python 3 has its own kind of file objects, independent of standard C and Readline. Python 3 only uses Readline if sys.stdin corresponds to the original C stdin FILE object.

Perhaps Python 3 could support Readline with other file objects (or at least file descriptors), but I think that would be a new feature.

So, if I understood correctly, the `readline` module only works for an unmodified `sys.stdin`, which is implemented in terms of C `FILE*` structures. Anything created by `open` will not be implemented in terms of C `FILE*`, and so all history and completion features of  `readline` will be (silently) disabled.

Is there any workaround to make `open` (or some other file-opening function) return a file object wrapping a C `FILE*`?

Maybe this behavior should be documented in the `readline` documentation? (https://docs.python.org/3/library/readline.html)

I agree it would be good to document when the Readline library is invoked. Yes, the “readline‭” module is only designed to work with the original sys.stdin and sys.stdout. Python’s “open” function does not use <stdio.h> FILE objects, but Python does use <stdio.h> FILE objects internally for its Readline hook, and passes them to the Readline library. Python falls back to dumber implementations, which may not implement history nor completion, when it decides not to use Readline.

The “readline” module is implemented in Modules/readline.c and uses “rl_instream” <https://cnswww.cns.cwru.edu/php/chet/readline/readline.html#IDX228> and “rl_outstream” to specify <stdio.h> FILE objects that the Readline library uses. These FILE objects are passed through the PyOS_ReadlineFunctionPointer hook <https://docs.python.org/3.4/c-api/veryhigh.html#c.PyOS_ReadlineFunctionPointer> from the PyOS_Readline function in Parser/myreadline.c. They are required to be terminals (checked with the Posix “isatty” call).

The implementation of Python’s “input” function is in the “builtin_input” C function in Python/bltinmodule.c. Before calling PyOS_Readline, it requires that sys.stdin.fileno() and sys.stdout.fileno() match the <stdio.h> stdin and stdout FILE objects. It also does its own isatty checks.

You might have some luck calling “fopen” with the “ctypes” module, or writing your own Readline module, but it wouldn’t be straightforward. You might be able to fool the check by reopening file descriptors 0 and 1, but that seems rather hacky. Or you might rely on the OS to provide history and/or completion (I think the Windows console does this in a limited way), or an external Readline wrapper program (e.g. search for “rlwrap”).