classification
Title: input() prints to original stdout even if sys.stdout is wrapped
Type: Stage:
Components: Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Drekin, martin.panter, wiggin15
Priority: normal Keywords:

Created on 2016-10-06 09:13 by wiggin15, last changed 2016-10-10 07:29 by Drekin.

Messages (5)
msg278179 - (view) Author: Arnon Yaari (wiggin15) * Date: 2016-10-06 09:13
When I wrap sys.stdout with a custom object that overrides the 'write' method, regular prints use the custom write method, but the input() function prints the prompt to the original stdout.
This is broken on Python 3.5. Earlier versions work correctly.
In the following example 'write' does nothing, so I expect no output, but the input() function outputs to stdout anyway:

import sys

class StreamWrapper(object):
    def __init__(self, wrapped):
        self.__wrapped = wrapped

    def __getattr__(self, name):
        # 'write' is overridden but for every other function, like 'flush', use the original wrapped stream
        return getattr(self.__wrapped, name)

    def write(self, text):
        pass

orig_stdout = sys.stdout
sys.stdout = StreamWrapper(orig_stdout)
print('a')   # this prints nothing
input('b')   # this should print nothing, but prints 'b' (in Python 3.5 and up only)

Looks like this was broken in http://bugs.python.org/issue24402 . Adding the 'fileno' function from this issue fixes the problem, but it's just a workaround. This affects the colorama package: https://github.com/tartley/colorama/issues/103
msg278181 - (view) Author: Adam Bartoš (Drekin) * Date: 2016-10-06 10:23
A related issue is that the REPL doesn't use sys.stdin for input, see #17620. Another related issue is #28333. I think that the situation around stdio in Python is complicated an inflexible (by stdio I mean all the interactions between REPL, input(), print(), sys.std* streams, sys.displayhook, sys.excepthook, C-level readline hooks). It would be nice to tidy up these interactions and document them at one place.

Currently, input() tries to detect whether sys.stdin and sys.stdout are interactive and have the right filenos, and handles the cases different way. I propose input() to be a thin wrapper (stripping a newline, generating EOFError) around proposed sys.readlinehook(). By default, sys.readlinehook would be GNU readline on Unix and stdio_readline (which just uses sys.stdout and sys.stdin) on Windows. I think that would fix all the problems like this one and changing/wrapping sys.std* streams would just work.

My proposal is at https://mail.python.org/pipermail/python-dev/2015-November/142246.html and there is discission at #17620. Recently, the related issue #1602 was fixed and there is hope there will be progress with #17620.
msg278183 - (view) Author: Adam Bartoš (Drekin) * Date: 2016-10-06 10:29
Other related issues are #1927 and #24829.
msg278394 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-10-09 23:57
From memory, there are at least three code paths for input():

1. Fallback implementation, used when stdout is a pipe or other non-terminal
2. Default PyOS_ReadlineFunctionPointer() hook, used when stdout is a terminal: https://docs.python.org/3.5/c-api/veryhigh.html#c.PyOS_ReadlineFunctionPointer
3. Gnu Readline (or another hook installed by a C module)

Arnon’s problem only seems to occur in the last two cases, when PyOS_ReadlineFunctionPointer() is used. The problem is that PyOS_ReadlineFunctionPointer() uses C <stdio.h> FILE pointers, not Python file objects. Python calls sys.stdout.fileno() to see if it can substitute the C-level stdout FILE object.

Adam: I think your sys.readlinehook() proposal is independent of Arnon’s problem. We would still need some logic to decide what to pass to libraries like Gnu Readline that want a FILE pointer instead of a Python file object.

The fileno() method is documented all file objects, including text files like sys.stdout. So I think implementing your own fileno() method is completely valid.

One other idea that comes to mind is if Python checked if the sys.stdout object has been changed (e.g. sys.stdout != sys.__stdout__), rather than just comparing fileno() values. But I am not sure if this change is worth it.

BTW if I revert the fix for Issue 24402 (I also tried 3.3.3), the problem occurs even when a custom fileno() is defined. On the other hand, Python 2’s raw_input() is not affected, presumably because it uses PyFile_AsFile() and fails immediately if stdout is a custom class.
msg278406 - (view) Author: Adam Bartoš (Drekin) * Date: 2016-10-10 07:29
Does GNU readline do anything fancy about printing the prompt? Because you may want to use GNU readline for autocompletition while still enable colored output via wrapped stdout. Both at the same time with one call to input(). It seems that currently either you go interactive and ignore sys.std*, or you lose readline capabilities.
History
Date User Action Args
2016-10-10 07:29:36Drekinsetmessages: + msg278406
2016-10-09 23:57:14martin.pantersetnosy: + martin.panter
messages: + msg278394
2016-10-06 10:29:17Drekinsetmessages: + msg278183
2016-10-06 10:23:49Drekinsetnosy: + Drekin
messages: + msg278181
2016-10-06 09:13:03wiggin15create