Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interactive interpreter doesn't flush stderr prompty #65624

Closed
marczellm mannequin opened this issue May 4, 2014 · 19 comments
Closed

Interactive interpreter doesn't flush stderr prompty #65624

marczellm mannequin opened this issue May 4, 2014 · 19 comments
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@marczellm
Copy link
Mannequin

marczellm mannequin commented May 4, 2014

BPO 21425
Nosy @ncoghlan, @pitrou, @tjguk, @skrah, @davidmalcolm, @zware, @eryksun
Files
  • flush_stderr.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2014-06-12.09:30:01.001>
    created_at = <Date 2014-05-04.08:28:40.048>
    labels = ['interpreter-core', 'type-bug']
    title = "Interactive interpreter doesn't flush stderr prompty"
    updated_at = <Date 2014-06-12.09:30:01.000>
    user = 'https://bugs.python.org/marczellm'

    bugs.python.org fields:

    activity = <Date 2014-06-12.09:30:01.000>
    actor = 'berker.peksag'
    assignee = 'none'
    closed = True
    closed_date = <Date 2014-06-12.09:30:01.001>
    closer = 'berker.peksag'
    components = ['Interpreter Core']
    creation = <Date 2014-05-04.08:28:40.048>
    creator = 'marczellm'
    dependencies = []
    files = ['35206']
    hgrepos = []
    issue_num = 21425
    keywords = ['patch']
    message_count = 19.0
    messages = ['217861', '218218', '218219', '218252', '218261', '218262', '218264', '218265', '218266', '218267', '218269', '218271', '218272', '218274', '218276', '218277', '218280', '218281', '218282']
    nosy_count = 11.0
    nosy_names = ['ncoghlan', 'pitrou', 'tim.golden', 'Arfrever', 'skrah', 'dmalcolm', 'python-dev', 'zach.ware', 'eryksun', 'David.Edelsohn', 'marczellm']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue21425'
    versions = ['Python 3.4', 'Python 3.5']

    @marczellm
    Copy link
    Mannequin Author

    marczellm mannequin commented May 4, 2014

    When I run a Python 3.3.4 prompt inside Emacs 24.3 on Windows 7, correct commands are evaluated immediately, but incorrect ones are delayed (I have to press Enter one more time), as seen below:

        >>> 1
        1
        >>> nonsense
        >>>
        Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        NameError: name 'nonsense' is not defined

    Python 2 does not do this. I've filed an Emacs bug report (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17304) and got the response to try this on the command line (where cat.exe is from an MSYS installation):

    python 2>&1 | cat.exe
    

    and it behaves the same way as in Emacs.

    @marczellm marczellm mannequin added OS-windows type-bug An unexpected behavior, bug, or error labels May 4, 2014
    @pitrou
    Copy link
    Member

    pitrou commented May 10, 2014

    This can be reproduced as easily under Linux:

    $ ./python 2>&1 | cat
    Python 3.5.0a0 (default:17689e43839a+39f2a78f4357+, May  9 2014, 00:30:19) 
    [GCC 4.8.1] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> 5
    5
    >>> 1/0
    >>> nonsense
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ZeroDivisionError: division by zero
    >>> 
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'nonsense' is not defined
    >>>

    @pitrou pitrou added interpreter-core (Objects, Python, Grammar, and Parser dirs) and removed OS-windows labels May 10, 2014
    @pitrou pitrou changed the title Python 3 pipe handling breaks python mode in emacs on Windows Interactive interpreter doesn't flush stderr prompty May 10, 2014
    @pitrou
    Copy link
    Member

    pitrou commented May 10, 2014

    Attached patch fixes it under Linux, and adds tests.
    I'd be glad if someone could give it a run under Windows.

    @eryksun
    Copy link
    Contributor

    eryksun commented May 11, 2014

    The patch fixes python 2>&1 | cat.exe for me on Windows, where cat.exe is from GnuWin32. The tests also pass. It looks like failing would entail hanging until a timeout.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 11, 2014

    New changeset ab3e012c45d0 by Antoine Pitrou in branch '3.4':
    Issue bpo-21425: Fix flushing of standard streams in the interactive interpreter.
    http://hg.python.org/cpython/rev/ab3e012c45d0

    New changeset d1c0cf44160c by Antoine Pitrou in branch 'default':
    Issue bpo-21425: Fix flushing of standard streams in the interactive interpreter.
    http://hg.python.org/cpython/rev/d1c0cf44160c

    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    Thanks for the report. This should now be fixed in 3.4 and 3.5.

    @pitrou pitrou closed this as completed May 11, 2014
    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    Apparently the new tests hang on the "Fedora without threads" buildbot.
    Stefan, could you take a look?

    http://buildbot.python.org/all/builders/AMD64%20Fedora%20without%20threads%203.4/builds/123

    @pitrou pitrou reopened this May 11, 2014
    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    Judging by the faulthandler traceback in the PowerLinux build:

    Timeout (1:00:00)!
    Thread 0x0000008074e46670 (most recent call first):
    File "/home/shager/cpython-buildarea/3.4.edelsohn-powerlinux-ppc64/build/Lib/test/test_cmd_line_script.py", line 192 in interactive_python
    File "/home/shager/cpython-buildarea/3.4.edelsohn-powerlinux-ppc64/build/Lib/contextlib.py", line 59 in __enter__
    File "/home/shager/cpython-buildarea/3.4.edelsohn-powerlinux-ppc64/build/Lib/test/test_cmd_line_script.py", line 205 in check_repl_stderr_flush
    File "/home/shager/cpython-buildarea/3.4.edelsohn-powerlinux-ppc64/build/Lib/test/test_cmd_line_script.py", line 220 in test_repl_stderr_flush
    [snip]

    ... the test is actually stuck trying to read the interactive prompt from stderr:

                # Drain stderr until prompt
                while True:
                    data = stderr.read(4)
                    if data == b">>> ":
                        break
                    stderr.readline()            <-- HERE

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 11, 2014

    New changeset ffae7aad9dfa by Antoine Pitrou in branch 'default':
    Add debugging output for bpo-21425
    http://hg.python.org/cpython/rev/ffae7aad9dfa

    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    So, it seems like under Fedora (or PowerLinux), the Python interactive prompt produces different byte contents on stderr:

    [ 21/389] test_cmd_line_script
    repl stderr[:4]: b'Pyth'
    repl stderr[:4]: b'[GCC'
    repl stderr[:4]: b'Type'
    repl stderr[:4]: b'\x1b[?1'

    I wonder if it's because of GNU readline?

    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 11, 2014

    New changeset e57718ac8ff8 by Antoine Pitrou in branch 'default':
    Try workaround for test issues in bpo-21425
    http://hg.python.org/cpython/rev/e57718ac8ff8

    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    According to https://bugzilla.redhat.com/show_bug.cgi?id=880393, this has actually been reported to us as bpo-19884.

    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    Workarounds seem to work.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 11, 2014

    New changeset 58e2116576cf by Antoine Pitrou in branch '3.4':
    Try workaround for test issues in bpo-21425
    http://hg.python.org/cpython/rev/58e2116576cf

    @pitrou pitrou closed this as completed May 11, 2014
    @pitrou
    Copy link
    Member

    pitrou commented May 11, 2014

    Workaround actually broke tests under shared library builds, because LD_LIBRARY_PATH isn't forwarded anymore. Will try another fix :-(

    @pitrou pitrou reopened this May 11, 2014
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 11, 2014

    New changeset 5a1b2566d68e by Antoine Pitrou in branch 'default':
    Try to fix issue bpo-21425 workaround for shared library builds
    http://hg.python.org/cpython/rev/5a1b2566d68e

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 11, 2014

    New changeset 974c0718c7e0 by Antoine Pitrou in branch '3.4':
    Try to fix issue bpo-21425 workaround for shared library builds
    http://hg.python.org/cpython/rev/974c0718c7e0

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants