This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add support for path-like objects to multiprocessing.set_executable for Windows to be on a par with Unix-like systems
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: maggyero
Priority: normal Keywords: patch

Created on 2022-02-11 14:57 by maggyero, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 31279 open maggyero, 2022-02-11 14:57
Messages (1)
msg413073 - (view) Author: Géry (maggyero) * Date: 2022-02-11 14:57
Any [path-like object](https://docs.python.org/3/glossary.html) can be passed to `multiprocessing.set_executable`, i.e. objects with `str`, `bytes`, or `os.PathLike` type.

For instance these work (tested on MacOS with all start methods: ‘spawn’, ‘fork’, and ‘forkserver’):

- `multiprocessing.set_executable(sys.executable)` (`str`);
- `multiprocessing.set_executable(sys.executable.encode())` (`bytes`);
- `multiprocessing.set_executable(pathlib.Path(sys.executable))` (`os.PathLike`).

This is because the ‘fork’ start method does not exec any program in the subprocess, the ‘spawn’ start method converts its path argument to `bytes` with `os.fsencode` before passing to [`_posixsubprocess.fork_exec`](https://github.com/python/cpython/blob/v3.10.2/Lib/multiprocessing/util.py#L452-L455), and the ‘forkserver’ start method spawns a server process (like with the ‘spawn’ start method) which then forks itself at each request (like the ‘fork’ start method):

```
        return _posixsubprocess.fork_exec(
            args, [os.fsencode(path)], True, passfds, None, None,
            -1, -1, -1, -1, -1, -1, errpipe_read, errpipe_write,
            False, False, None, None, None, -1, None)
```

Linux (and other Unix-like systems) uses the same code than MacOS for the three start methods so it should work for it too.

However I have not tested this on Windows which uses the function [`_winapi.CreateProcess`](https://github.com/python/cpython/blob/v3.10.2/Lib/multiprocessing/popen_spawn_win32.py#L73-L75) for the ‘spawn’ start method (the only start method available on this OS) but I noticed that no conversion to `str` (not to `bytes` this time, since [the function expects `str`](https://github.com/python/cpython/blob/v3.10.2/Modules/_winapi.c#L1049)) of the path argument with `os.fsdecode` (not `os.fsencode` this time) is performed before passing it to the function:

```
                hp, ht, pid, tid = _winapi.CreateProcess(
                    python_exe, cmd,
                    None, None, False, 0, env, None, None)
```

So on Windows only `str` path can be passed to `multiprocessing.set_executable`. This PR fixes this to be on a par with Unix-like systems which accept any path-like objects.
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 90876
2022-02-11 22:25:00maggyerosettitle: Add support of path-like objects to multiprocessing.set_executable for Windows to match Unix-like systems -> Add support for path-like objects to multiprocessing.set_executable for Windows to be on a par with Unix-like systems
2022-02-11 16:22:47maggyerosettype: enhancement
2022-02-11 14:57:24maggyerosetkeywords: + patch
stage: patch review
pull_requests: + pull_request29439
2022-02-11 14:57:09maggyerocreate