This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Use CreateProcessW for Python 2.7 on Windows.
Type: enhancement Stage: resolved
Components: Versions:
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: Ray Donnelly, giampaolo.rodola, gregory.p.smith, josh.r, paul.moore, serhiy.storchaka, steve.dower, terry.reedy, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2019-03-18 17:45 by Ray Donnelly, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
0017-Use-CreateProcessW-to-support-Unicode.patch Ray Donnelly, 2019-03-18 17:45 First attempt at supporting unicode in _subprocess.c
Messages (6)
msg338270 - (view) Author: Ray Donnelly (Ray Donnelly) * Date: 2019-03-18 17:45
Hi all,

I'd like to entertain some discussion around the idea of calling CreateProcessW instead of CreateProcess on Windows.

I've written a patch as a proof of concept and I would love to get some feedback. I guess I've broken the normal ACP/getfilesystemencoding() expectation for byte strings here. My idea to fix this was to use CreateProcessW only when all arguments (program name, arguments, cwd, environment) are unicode already.

The reason we'd like to use it on Anaconda Distribution is that we would like for conda to be able to handle Unicode as well as possible in as many situations as possible, including running a Python2 conda and creating conda envs with all sorts of glyphs in it.

We run into bug reports quite frequently from people who try to install Miniconda2 or Anaconda2 in their home directories due to their username containing certain codepoints.
msg338271 - (view) Author: Ray Donnelly (Ray Donnelly) * Date: 2019-03-18 17:47
.. and alternative to my ACP idea would be to use `GetACP()` or `getfilesystemencoding()` .. or? Suggestions welcome!
msg338635 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-03-23 00:31
Is CreateProcessW already used in current 3.x?
msg338640 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2019-03-23 02:52
2.7 is in bug fix only mode, and even that ends at the end of this year. This seems more like a feature request than an actual bug; it works, but not great with Unicode in some cases (which describes large parts of Python 2).

It does look like Python 3.7 at least is using CreateProcessW already ( https://github.com/python/cpython/blob/3.7/Modules/_winapi.c#L1062 ), going along with the generally more Unicode-friendly vibe there.

The subprocess32 module on PyPI performed backports from Py3's subprocess to Py2 for POSIX systems; perhaps talk to the author (gregory.p.smith, whom I have nosied) about backporting similar improvements for the Windows side of the aisle?
msg338649 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-03-23 06:58
It is not enough to just replace CreateProcess with CreateProcessW. You will need to change also other related calls and resolve issues when you pass some arguments as 8-bit strings and other arguments as Unicode strings. You will need to backport a half of Python 3. At end, this will break someones code.

The correct way to solve this issue -- upgrade to Python 3, where these issues were solved more smoothly and contsistently.
msg338651 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-03-23 07:39
2.7 was closed to new features eons ago.

While subprocess32 backport might be a plausible home for this, I really can't handle doing anything significant for Windows within the confines of that project (it already makes me nervous that anyone is using subprocess32 on Windows at all).  

I don't intend to update subprocess32 with any new features at this point.

If you need an enhanced subprocess module for use on 2.7 on Windows I suggest creating your own fork/package for use in Anaconda 2.x distributions.
History
Date User Action Args
2022-04-11 14:59:12adminsetgithub: 80535
2019-03-23 07:39:43gregory.p.smithsetstatus: open -> closed
type: enhancement
messages: + msg338651

resolution: rejected
stage: resolved
2019-03-23 06:58:13serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg338649
2019-03-23 02:52:01josh.rsetnosy: + josh.r, gregory.p.smith
messages: + msg338640
2019-03-23 00:31:15terry.reedysetnosy: + terry.reedy
messages: + msg338635
2019-03-18 17:47:32Ray Donnellysetmessages: + msg338271
2019-03-18 17:45:50Ray Donnellycreate