Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Popen doesn't work on Windows when args is a list #76945

Closed
serhiy-storchaka opened this issue Feb 4, 2018 · 7 comments
Closed

Popen doesn't work on Windows when args is a list #76945

serhiy-storchaka opened this issue Feb 4, 2018 · 7 comments
Assignees
Labels
3.7 (EOL) end of life 3.8 only security fixes OS-windows stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 32764
Nosy @gpshead, @pfmoore, @tjguk, @zware, @serhiy-storchaka, @zooba, @Phaqui
PRs
  • bpo-32764: Fixes test_nonexisting_with_pipes using incorrect test value #5762
  • Superseder
  • bpo-30121: Windows: subprocess debug assertion on failure to execute the process
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/zooba'
    closed_at = <Date 2018-02-20.00:34:13.344>
    created_at = <Date 2018-02-04.16:01:42.348>
    labels = ['3.7', '3.8', 'type-bug', 'library', 'OS-windows']
    title = "Popen doesn't work on Windows when args is a list"
    updated_at = <Date 2018-02-20.00:34:13.342>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2018-02-20.00:34:13.342>
    actor = 'steve.dower'
    assignee = 'steve.dower'
    closed = True
    closed_date = <Date 2018-02-20.00:34:13.344>
    closer = 'steve.dower'
    components = ['Library (Lib)', 'Windows']
    creation = <Date 2018-02-04.16:01:42.348>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 32764
    keywords = ['patch']
    message_count = 7.0
    messages = ['311603', '311607', '311609', '311613', '312383', '312385', '312386']
    nosy_count = 7.0
    nosy_names = ['gregory.p.smith', 'paul.moore', 'tim.golden', 'zach.ware', 'serhiy.storchaka', 'steve.dower', 'Phaqui']
    pr_nums = ['5762']
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '30121'
    type = 'behavior'
    url = 'https://bugs.python.org/issue32764'
    versions = ['Python 3.7', 'Python 3.8']

    @serhiy-storchaka
    Copy link
    Member Author

    test_subprocess is failing on Windows.

    C:\py\cpython3.7>./python -m test -uall -v -m test_nonexisting_with_pipes test_subprocess
    Running Debug|Win32 interpreter...
    == CPython 3.7.0b1+ (heads/3.7:1a0239e, Feb 4 2018, 16:19:37) [MSC v.1911 32 bit (Intel)]
    == Windows-10-10.0.16299-SP0 little-endian
    == cwd: C:\py\cpython3.7\build\test_python_11092
    == CPU count: 2
    == encodings: locale=cp1251, FS=utf-8
    Run tests sequentially
    0:00:00 [1/1] test_subprocess
    test_nonexisting_with_pipes (test.test_subprocess.ProcessTestCase) ... FAIL
    test_nonexisting_with_pipes (test.test_subprocess.ProcessTestCaseNoPoll) ... skipped 'Test needs selectors.PollSelector'

    ======================================================================
    FAIL: test_nonexisting_with_pipes (test.test_subprocess.ProcessTestCase)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "C:\py\cpython3.7\lib\test\test_subprocess.py", line 1194, in test_nonexisting_with_pipes
        self.assertEqual(stderr, "")
    AssertionError: 'Traceback (most recent call last):\n  Fil[923 chars]le\n' != ''
    Diff is 965 characters long. Set self.maxDiff to None to see it.

    Ran 2 tests in 0.171s

    FAILED (failures=1, skipped=1)
    test test_subprocess failed
    test_subprocess failed

    1 test failed:
    test_subprocess

    Total duration: 391 ms
    Tests result: FAILURE

    Here stderr is:

    Traceback (most recent call last):
      File "C:\py\cpython3.7\lib\subprocess.py", line 1101, in _execute_child
        args = os.fsdecode(args)  # os.PathLike -> str
      File "C:\py\cpython3.7\\lib\os.py", line 821, in fsdecode
        filename = fspath(filename)  # Does type-checking of `filename`.
    TypeError: expected str, bytes or os.PathLike object, not list
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
      File "C:\py\cpython3.7\lib\subprocess.py", line 756, in __init__
        restore_signals, start_new_session)
      File "C:\py\cpython3.7\lib\subprocess.py", line 1104, in _execute_child
        args[0] = os.fsdecode(args[0])  # os.PathLike -> str
      File "C:\py\cpython3.7\\lib\os.py", line 821, in fsdecode
        filename = fspath(filename)  # Does type-checking of `filename`.
    TypeError: expected str, bytes or os.PathLike object, not tuple

    In _execute_child args is passed to os.fsdecode() unless it is a string. In this case args is a list. os.fsdecode() doesn't accept a list.

    The regression was introduced in bpo-31961.

    @serhiy-storchaka serhiy-storchaka added 3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir OS-windows type-bug An unexpected behavior, bug, or error labels Feb 4, 2018
    @phaqui
    Copy link
    Mannequin

    phaqui mannequin commented Feb 4, 2018

    This is strange, because _execute_child calls os.fsdecode with args as the argument, which may be a list. os.fsdecode calls fspath. Now, the python docstring of _fspath, as defined in Lib/os.py on line 1031, clearly states that it will raise a TypeError if the argument is not of type bytes, str or is a os.PathLike object, and that's probably why I wrote the initial code the way I did (catching TypeError from os.fsdecode).

    Doesn't the try-except block actually catch this TypeError? I don't understand off the top of my head why my code doesn't catch this exception..

    @phaqui
    Copy link
    Mannequin

    phaqui mannequin commented Feb 4, 2018

    Also, isn't there continuous integration testing? Everything passed on the PR, so where does this come from?

    @phaqui
    Copy link
    Mannequin

    phaqui mannequin commented Feb 4, 2018

    Wait a minute. The failing test is test_nonexisting_with_pipes, and it fails because args[0] is a tuple - how can that be? Nobody is supposed to pass cmd=sequence-where-first-element-is-a-tuple!

    Is everything all right with the test itself?

    @gpshead gpshead self-assigned this Feb 6, 2018
    @zooba
    Copy link
    Member

    zooba commented Feb 19, 2018

    Turns out this is a trivial typo in the test, so I stole the issue from Greg and pushed a PR.

    @zooba zooba assigned zooba and unassigned gpshead Feb 19, 2018
    @zware
    Copy link
    Member

    zware commented Feb 20, 2018

    I fixed this independently earlier today, tied to bpo-30121 (the issue that introduced the test bug) because I didn't find this one. See PR5758, PR5759 (3.7), and PR5760 (3.6).

    @zooba
    Copy link
    Member

    zooba commented Feb 20, 2018

    Sounds good to me

    @zooba zooba closed this as completed Feb 20, 2018
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes OS-windows stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants