Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

input() uses sys.__stdout__ instead of sys.stdout for prompt #68590

Closed
KeitaKita mannequin opened this issue Jun 7, 2015 · 15 comments
Closed

input() uses sys.__stdout__ instead of sys.stdout for prompt #68590

KeitaKita mannequin opened this issue Jun 7, 2015 · 15 comments
Assignees
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@KeitaKita
Copy link
Mannequin

KeitaKita mannequin commented Jun 7, 2015

BPO 24402
Nosy @pitrou, @taleinat, @berkerpeksag, @vadmium, @serhiy-storchaka, @eryksun, @zhangyangyu
Files
  • issue24402.fix_only.patch: patch with just a fix (no tests)
  • input-terminal.patch: Now with test case
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/vadmium'
    closed_at = <Date 2015-10-10.10:04:49.822>
    created_at = <Date 2015-06-07.09:30:52.044>
    labels = ['interpreter-core', 'type-bug']
    title = 'input() uses sys.__stdout__ instead of sys.stdout for prompt'
    updated_at = <Date 2015-10-10.10:04:49.821>
    user = 'https://bugs.python.org/KeitaKita'

    bugs.python.org fields:

    activity = <Date 2015-10-10.10:04:49.821>
    actor = 'martin.panter'
    assignee = 'martin.panter'
    closed = True
    closed_date = <Date 2015-10-10.10:04:49.822>
    closer = 'martin.panter'
    components = ['Interpreter Core']
    creation = <Date 2015-06-07.09:30:52.044>
    creator = 'Keita Kita'
    dependencies = []
    files = ['39697', '40707']
    hgrepos = []
    issue_num = 24402
    keywords = ['patch']
    message_count = 15.0
    messages = ['244949', '244957', '245279', '245280', '252436', '252455', '252458', '252676', '252677', '252683', '252686', '252688', '252689', '252690', '252699']
    nosy_count = 9.0
    nosy_names = ['pitrou', 'taleinat', 'python-dev', 'berker.peksag', 'martin.panter', 'serhiy.storchaka', 'eryksun', 'xiang.zhang', 'Keita Kita']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue24402'
    versions = ['Python 3.4', 'Python 3.5', 'Python 3.6']

    @KeitaKita
    Copy link
    Mannequin Author

    KeitaKita mannequin commented Jun 7, 2015

    On Python 3.4, input() does not use wrapped sys.stdout. For example, the following script should print "Wrapped stdout" by print() and input() because sys.stdout is replaced with Writer object.

    --------

    import io
    import sys
    
    
    class Writer:
        def __getattr__(self, name):
            return getattr(sys.__stdout__, name)
    
        def write(self, data):
            if data != '\n':
                sys.__stdout__.write('Wrapped stdout\n')
    
        def fileno():
            raise OSError()
    
    
    sys.stdout = Writer()
    
    print('print')
    input('input')

    But the script does not print "Wrapped stdout" as prompt of input(). The script
    prints the following.

    -----
    Wrapped stdout
    input
    ----

    Although, when sys.stdin is also wrapped, input() will use sys.stdout for prompt. For example, the following script prints "Wrapped stdout" by print() and input().

    ----

    import io
    import sys
    
    
    class Writer:
        def __getattr__(self, name):
            return getattr(sys.__stdout__, name)
    
        def write(self, data):
            if data != '\n':
                sys.__stdout__.write('Wrapped stdout\n')
    
        def fileno():
            raise OSError()
    
    
    class Reader:
        def __getattr__(self, name):
            return getattr(sys.__stdin__, name)
    
        def read(self, size):
            return sys.__stdin__.read(size)
    
        def fileno():
            raise OSError()
    
    
    sys.stdout = Writer()
    sys.stdin = Reader()
    
    print('print')
    input('input')

    The script prints the following.

    -----
    Wrapped stdout
    Wrapped stdout
    ----

    @KeitaKita KeitaKita mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Jun 7, 2015
    @eryksun
    Copy link
    Contributor

    eryksun commented Jun 7, 2015

    This looks like an oversight. In the implementation in bltinmodule.c, if checking sys.stdout.fileno() fails, it clears the error without setting tty = 0. It's the same in 3.5.

        /* We should only use (GNU) readline if Python's sys.stdin and
           sys.stdout are the same as C's stdin and stdout, because we
           need to pass it those. */
        tmp = _PyObject_CallMethodId(fin, &PyId_fileno, "");
        if (tmp == NULL) {
            PyErr_Clear();
            tty = 0;
        }
        else {
            fd = PyLong_AsLong(tmp);
            Py_DECREF(tmp);
            if (fd < 0 && PyErr_Occurred())
                return NULL;
            tty = fd == fileno(stdin) && isatty(fd);
        }
        if (tty) {
            tmp = _PyObject_CallMethodId(fout, &PyId_fileno, "");
            if (tmp == NULL)
                PyErr_Clear();
            else {
                fd = PyLong_AsLong(tmp);
                Py_DECREF(tmp);
                if (fd < 0 && PyErr_Occurred())
                    return NULL;
                tty = fd == fileno(stdout) && isatty(fd);
            }
        }

    @taleinat
    Copy link
    Contributor

    Okay, so the change is straightforward. With this change the entire test suite passes on my machine.

    We'll need a test for this for the test suite, and that's far less straightforward. It seems that test.test_builtin.BuiltinTest.check_input_tty() could be a starting point. Could someone come up with a working test for this?

    @taleinat
    Copy link
    Contributor

    Here's a patch with just the required fix (no tests).

    @vadmium
    Copy link
    Member

    vadmium commented Oct 6, 2015

    Patch itself looks good.

    @vadmium
    Copy link
    Member

    vadmium commented Oct 7, 2015

    This patch includes a test case using pty.fork() to ensure stdin is a terminal, so the bug would be triggered. It is a bit tricky, but hopefully works on Unix platforms. I have only tested it on Linux.

    @serhiy-storchaka
    Copy link
    Member

    LGTM. See also minor note on Rietveld.

    I think it is worth to extract all input() tests to separate class.

    @vadmium
    Copy link
    Member

    vadmium commented Oct 10, 2015

    Okay I plan to commit a modified version where I moved the input() test methods that use pseudo terminals to a new PtyTests class. The test_input() method stays in the original class, because it shares a helper method with other tests.

    I am resisting dropping that decode() step, because it makes any error message from the forked child hard to read:

    AssertionError: 256 != 0 : b'quux\r\nTraceback (most recent call last):\r\n  File "/media/disk/home/proj/python/cpython/Lib/test/test_builtin.py", line 1593, in test_input_no_stdout_fileno\r\n    input("prompt")\r\nTypeError: bad argument type for built-in operation\r\n'

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 10, 2015

    New changeset 425d81d10b13 by Martin Panter in branch '3.4':
    Issue bpo-24402: Fix input() when stdout.fileno() fails; diagnosed by Eryksun
    https://hg.python.org/cpython/rev/425d81d10b13

    New changeset e97d940a6543 by Martin Panter in branch '3.5':
    Issue bpo-24402: Merge input() fix from 3.4 into 3.5
    https://hg.python.org/cpython/rev/e97d940a6543

    New changeset bcc0f8eb6797 by Martin Panter in branch 'default':
    Issue bpo-24402: Merge input() fix from 3.5
    https://hg.python.org/cpython/rev/bcc0f8eb6797

    @vadmium vadmium closed this as completed Oct 10, 2015
    @vadmium
    Copy link
    Member

    vadmium commented Oct 10, 2015

    Seems my test hangs the Free BSD 9 buildbot, although Free BSD 10 apparently passed.

    http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.4/builds/1128/steps/test/logs/stdio

    [390/390] test_builtin
    Timeout (1:00:00)!
    Thread 0x0000000801407400 (most recent call first):
    File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/test/test_builtin.py", line 1585 in test_input_no_stdout_fileno
    File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/unittest/case.py", line 580 in run
    File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/unittest/case.py", line 628 in __call__

    @vadmium vadmium reopened this Oct 10, 2015
    @zhangyangyu
    Copy link
    Member

    This test also fails on Ubuntu.

    [1/1] test_builtin
    test test_builtin failed -- Traceback (most recent call last):
      File "/home/angwer/my-cpython/Lib/test/test_builtin.py", line 1588, in test_input_no_stdout_fileno
        self.assertEqual(status, 0, output)
    AssertionError: 256 != 0 : quux
    Traceback (most recent call last):
      File "/home/angwer/my-cpython/Lib/test/test_builtin.py", line 1593, in test_input_no_stdout_fileno
        input("prompt")
    TypeError: bad argument type for built-in operation

    1 test failed:
    test_builtin

    @vadmium
    Copy link
    Member

    vadmium commented Oct 10, 2015

    I wrote this on Arch Linux, so I doubt your Ubuntu failure is valid. The error you see is what I would expect if the bug were not actually fixed. I suspect you need to recompile /Python/bltinmodule.c, which contains the bug fix.

    However, the test also hangs OS X buildbots. I can only guess at why it is hanging, either at the waitpid() line or the os.read() line. I am looking at refactoring my test to use the existing check_input_tty() infrastructure. It seems over the top by adding a separate pipe back to the parent process, but presumably will workaround whatever is causing the hang.

    @zhangyangyu
    Copy link
    Member

    It's my fault. I forget to recompile. Now it works.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 10, 2015

    New changeset d8dd9015b086 by Martin Panter in branch '3.4':
    Issue bpo-24402: Factor out PtyTests.run_child() in input() tests
    https://hg.python.org/cpython/rev/d8dd9015b086

    New changeset 6a8f96b46dce by Martin Panter in branch '3.5':
    Issue bpo-24402: Merge potential test fix from 3.4 into 3.5
    https://hg.python.org/cpython/rev/6a8f96b46dce

    New changeset cb574ee7231e by Martin Panter in branch 'default':
    Issue bpo-24402: Merge potential test fix from 3.5
    https://hg.python.org/cpython/rev/cb574ee7231e

    @vadmium
    Copy link
    Member

    vadmium commented Oct 10, 2015

    My second attempt seems to have avoided the hang.

    For the record, the backtrace from 3.4.koobs-freebsd9 did say it was hanging at the os.waitpid() line, but I still don’t understand why.

    @vadmium vadmium closed this as completed Oct 10, 2015
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants