classification
Title: Popen doesn't work on Windows when args is a list
Type: behavior Stage: resolved
Components: Library (Lib), Windows Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Windows: subprocess debug assertion on failure to execute the process
View: 30121
Assigned To: steve.dower Nosy List: Phaqui, gregory.p.smith, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2018-02-04 16:01 by serhiy.storchaka, last changed 2018-02-20 00:34 by steve.dower. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 5762 closed steve.dower, 2018-02-19 23:22
Messages (7)
msg311603 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-02-04 16:01
test_subprocess is failing on Windows.

C:\py\cpython3.7>./python -m test -uall -v -m test_nonexisting_with_pipes test_subprocess
Running Debug|Win32 interpreter...
== CPython 3.7.0b1+ (heads/3.7:1a0239e, Feb 4 2018, 16:19:37) [MSC v.1911 32 bit (Intel)]
== Windows-10-10.0.16299-SP0 little-endian
== cwd: C:\py\cpython3.7\build\test_python_11092
== CPU count: 2
== encodings: locale=cp1251, FS=utf-8
Run tests sequentially
0:00:00 [1/1] test_subprocess
test_nonexisting_with_pipes (test.test_subprocess.ProcessTestCase) ... FAIL
test_nonexisting_with_pipes (test.test_subprocess.ProcessTestCaseNoPoll) ... skipped 'Test needs selectors.PollSelector'

======================================================================
FAIL: test_nonexisting_with_pipes (test.test_subprocess.ProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\py\cpython3.7\lib\test\test_subprocess.py", line 1194, in test_nonexisting_with_pipes
    self.assertEqual(stderr, "")
AssertionError: 'Traceback (most recent call last):\n  Fil[923 chars]le\n' != ''
Diff is 965 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------
Ran 2 tests in 0.171s

FAILED (failures=1, skipped=1)
test test_subprocess failed
test_subprocess failed

1 test failed:
    test_subprocess

Total duration: 391 ms
Tests result: FAILURE


Here stderr is:

Traceback (most recent call last):
  File "C:\py\cpython3.7\lib\subprocess.py", line 1101, in _execute_child
    args = os.fsdecode(args)  # os.PathLike -> str
  File "C:\py\cpython3.7\\lib\os.py", line 821, in fsdecode
    filename = fspath(filename)  # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not list

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 16, in <module>
  File "C:\py\cpython3.7\lib\subprocess.py", line 756, in __init__
    restore_signals, start_new_session)
  File "C:\py\cpython3.7\lib\subprocess.py", line 1104, in _execute_child
    args[0] = os.fsdecode(args[0])  # os.PathLike -> str
  File "C:\py\cpython3.7\\lib\os.py", line 821, in fsdecode
    filename = fspath(filename)  # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not tuple

In _execute_child args is passed to os.fsdecode() unless it is a string. In this case args is a list. os.fsdecode() doesn't accept a list.

The regression was introduced in issue31961.
msg311607 - (view) Author: Anders Lorentsen (Phaqui) * Date: 2018-02-04 16:56
This is strange, because _execute_child calls os.fsdecode with `args` as the argument, which may be a list. os.fsdecode calls fspath. Now, the python docstring of _fspath, as defined in Lib/os.py on line 1031, clearly states that it will raise a TypeError if the argument is not of type bytes, str or is a os.PathLike object, and that's probably why I wrote the initial code the way I did (catching TypeError from os.fsdecode).

Doesn't the try-except block actually catch this TypeError? I don't understand off the top of my head why my code doesn't catch this exception..
msg311609 - (view) Author: Anders Lorentsen (Phaqui) * Date: 2018-02-04 17:11
Also, isn't there continuous integration testing? Everything passed on the PR, so where does this come from?
msg311613 - (view) Author: Anders Lorentsen (Phaqui) * Date: 2018-02-04 18:34
Wait a minute. The failing test is test_nonexisting_with_pipes, and it fails because args[0] is a tuple - how can that be? Nobody is supposed to pass cmd=sequence-where-first-element-is-a-tuple!

Is everything all right with the test itself?
msg312383 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-02-19 23:23
Turns out this is a trivial typo in the test, so I stole the issue from Greg and pushed a PR.
msg312385 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2018-02-20 00:31
I fixed this independently earlier today, tied to bpo-30121 (the issue that introduced the test bug) because I didn't find this one.  See PR5758, PR5759 (3.7), and PR5760 (3.6).
msg312386 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-02-20 00:34
Sounds good to me
History
Date User Action Args
2018-02-20 00:34:13steve.dowersetstatus: open -> closed
superseder: Windows: subprocess debug assertion on failure to execute the process
messages: + msg312386

resolution: duplicate
stage: patch review -> resolved
2018-02-20 00:31:53zach.waresetmessages: + msg312385
2018-02-19 23:23:15steve.dowersetassignee: gregory.p.smith -> steve.dower
messages: + msg312383
2018-02-19 23:22:26steve.dowersetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request5540
2018-02-06 16:54:36gregory.p.smithsetassignee: gregory.p.smith
2018-02-04 18:34:12Phaquisetmessages: + msg311613
2018-02-04 17:11:12Phaquisetmessages: + msg311609
2018-02-04 16:56:45Phaquisetmessages: + msg311607
2018-02-04 16:01:42serhiy.storchakacreate