GNU readline doesn't work if the C symbol "stdin" is
not a tty, even if the python symbol sys.stdin is. 
This will happen, for example, if a program initially
receives input from a pipe, but then changes sys.stdin
to read from /dev/tty.  (This is how programs like
"less" work)

Here's a sample program that exhibits this behavior:

------------------------------------------------------------------------
#!/usr/bin/env python

import sys

mail = sys.stdin.read ()

sys.stdin = open ('/dev/tty', 'r')

import readline

foo = raw_input ('add_idea> ')
print foo
------------------------------------------------------------------------

You can test this by saving the above program to a file
(foo.py), piping data to it, then trying to use GNU
readline editing commands at the prompt.

E.g.:

------------------------------------------------------------------------
liberty:~$ cat ideas.html | ./foo.py
add_idea> asdfsdf^Afoo
asdfsdffoo
------------------------------------------------------------------------

The patch attached seems to fix the problem.  You may
want to grep the source for other modules that may have
similar bugs.  Also, this patch assumes that the
readline module is imported *after* sys.stdin is
changed.  This much better than nothing (particularly
if it's documented), but there may be a better solution.

-- Agthorr

Logged In: YES 
user_id=6656

Comments:

1) in what ways does this change existing behaviour?  I can
think of a few, but are there any that will inconvenience
existing users
2) why not do the rl_instream = PySys_GetObject("stdin")
dance in call_readline()?

Logged In: YES 
user_id=6324

1) Well, it lets python treat sys.stdin as a tty even if C
stdin != python sys.stdin.  It still checks to make sure
sys.stdin is a tty using isatty().  If some user changes
sys.stdin to point to a tty, but *wants* Python to treat it
as a non-tty, then this might cause them some grief.  I
can't think of any case where they'd want to do that,
though.  The behavior would be unchanged when sys.stdin
points to a regular file.

2) hmm.. I suppose, ideally, the readline module should
smoothly handle sys.stdin being changed out from under it. 
Readline alters various terminal settings on rl_instream
during initialization, though.  For example, it changes the
terminal to raw or cbreak mode from cooked mode, so that it
can receive input a character at a time instead of a line at
a time.  It may be possible to uninitialized and
reinitialized terminal each time call_readline is called, I
suppose (I believe libreadline provides hooks for this).  It
would also have to check if sys.stdin is a tty, and call
PyFile_GetLine if it is not.

Logged In: YES 
user_id=6324

If I create a patch that operates as described in my
previous followup, will you apply it?  Is there anything I
can do to get this integrated into the main python trunk?  I
don't like having to repatch and rebuild python everytime a
new version comes out that I need for some other reason :>

Logged In: YES 
user_id=21627

Are you interested in pursuing this feature? I cannot promise 
that I will apply it blankly, only that I will review it.

Logged In: YES 
user_id=6324

Yes, I'm interested in pursuing it.  Let me know what I can
do to move this along.

Logged In: YES 
user_id=21627

I think you should implement a strategy to detect changes to
sys.stdin dynamically. For example, the old sys.stdin could
go away, closing the file, which would leave readline with a
garbage pointer.

Instead, I think you should hold onto sys.stdin, and compare
this reference from time to time with the current sys.stdin,
and arrange to reinitialize readline (if possible).

Logged In: YES 
user_id=6324

Okay, I've attached a new patch that works as you suggested.
 Let me know if you have any questions or concerns.

Logged In: YES 
user_id=21627

It appears that your patch can crash when sys.stdin/out is
not a file anymore:

>>> import readline
>>> import codecs,sys
>>> sys.stdout=codecs.getwriter('utf-8')(sys.stdout)
Segmentation fault (core dumped)

Logged In: YES 
user_id=6324

Jeez.  My apologies.

I will submit a new patch within a few days, and this time I
will do regression testing instead of just looking at the
problem I was trying to fix.

Logged In: YES 
user_id=6324

When in interactive mode, if the user changes
sys.std[in|out], would you agree that the interpreter should
continue using C std[in|out] to get commands? (this is the
current behavior without my patch)

The documentation for sys.std[in|out|err] seem to suggest
one should be able to redirect them, but I'm not sure that's
actually desirable.

(Your example only crashes if the interpreter is in
interactive mode.)

I've included another patch assuming this is the desired
behavior.  I tried several permutations of interactive vs
non-interactive, setting sys.stdin and/or sys.stdout to
None, and doing this before and after initializing readline.

Logged In: YES 
user_id=21627

Thanks for the patch, committed as

pythonrun.h 2.55;
readline.c 2.54;
myreadline.c 2.28;
tokenizer.c 2.68;
bltinmodule.c 2.266;
NEWS 1.503;
ACKS 2.212

The limitation (interpreter continues to read from stdin)
seems reasonable.

I suggest to reopen this issue as there was a regression with python3.

import sys
sys.stdin = open("/dev/tty", "r")
import readline
print(input())

Write some text and press left.
Expected: the cursor goes left.
Actual: prints '^[[D' as is readline had not been imported.

bltinmodule.c checks that the current sys.stdin filno matches the C stdin fileno. When they are different, it falls back to the default input implementation.

https://github.com/python/cpython/blob/1e7b858575d0ad782939f86aae4a2fa1c29e9f14/Python/bltinmodule.c#L2097

I noticed that PyFile_AsFile no longer exists. Would calling `fdopen` be acceptable?

Please, let me know if I should re-open a new bug for this one.

A workaround consists in replacing fd(0) with /dev/tty without modifying sys.stdin


import os

stdin = os.dup(0)
os.close(0)
tty = os.open("/dev/tty", os.O_RDONLY)
assert tty == 0

import readline
print("input:", input())
print("in:", os.read(stdin, 128))