classification
Title: Drop w9xpopen and all dependencies
Type: enhancement Stage: resolved
Components: Build, Windows Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Remove using of w9xopen in subprocess module
View: 14470
Assigned To: trent Nosy List: amaury.forgeotdarc, benrg, brian.curtin, christian.heimes, loewis, techtonik, trent
Priority: normal Keywords: patch

Created on 2008-03-18 20:09 by trent, last changed 2012-12-31 06:45 by brian.curtin. This issue is now closed.

Files
File name Uploaded Description Edit
deprecate-w9xpopen.patch amaury.forgeotdarc, 2010-11-14 21:20 review
Messages (8)
msg63978 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2008-03-18 20:09
Python 2.6+ drops support for Windows 95/98, which removes the need for
w9xpopen.  Get rid of the module and all dependencies (such as in the .msi).
msg64009 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-03-18 22:51
Tim Peters once commented that w9xpopen cannot go away as long as people 
still use alternative shells (through COMSPEC) that still have the 
original issue that command.com had.

I don't know how relevant that still is, and whether perhaps breakage is 
acceptable if people install odd shells that are not fully compatible with 
cmd.exe.
msg64249 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-03-21 18:21
I'm not sure either but I like to consider the removal of w9xpopen
wrapper for the 3.x series. The py3k project was started to remove old
cruft and I view w9xpopen as such a cruft.
msg121208 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-11-14 21:20
It is still time to add to 3.2 a DeprecationWarning when w9xpopen is used, and remove the feature in 3.3.
See attached patch
msg128894 - (view) Author: anatoly techtonik (techtonik) Date: 2011-02-20 06:27
I still see w9xpopen.exe in my Python 3.2 installation, which is kind of strange to me.
msg130298 - (view) Author: (benrg) Date: 2011-03-07 23:56
w9xpopen is currently used on NT. The patch to use it on NT was checked in by bquinlan in August of 2001 (http://mail.python.org/pipermail/patches/2001-August/005719.html). He claims that it is necessary in NT, even though (a) the cited knowledge base article explicitly states that it is not necessary on NT, and (b) the knowledge base article has now been deleted from Microsoft's web site, indicating that they consider it no longer relevant (they have deleted all Win9x-specific documentation, but Win2K-specific documentation is still there).

I just don't believe that the problem solved by w9xpopen has ever existed in any version of NT. There is no credible evidence for it. There are any number of other reasons why introducing an intermediate process might have hidden some unrelated bug or otherwise resolved the problem the Win9x->Win2K upgraders were having a decade ago. I think that the use of w9xpopen in NT is a bug, not an obsolete feature, and there's no reason it couldn't be gone in 3.2.1.

Also, I suppose it doesn't matter any more, but the logic for deciding when to run w9xpopen should be (target executable is 16-bit), which can be determined by reading the file header. Right now the test is (shell is True and (running on win9x or the command processor is named "command.com")). Every part of this test is deficient. Python programs can spawn 16-bit processes (including the shell itself) without using shell=True. Not every win9x shell is 16-bit; 32-bit shells like cmd.exe work fine. And there are 16-bit shells not named "command.com", such as 4DOS.
msg130305 - (view) Author: (benrg) Date: 2011-03-08 04:36
It turns out that, on Windows 7 32-bit with COMSPEC pointing to command.com, platform.popen('dir').read() works with w9xpopen and fails (no output) without it.

But the reason has nothing to do with the old Win9x problem. It's because subprocess always quotes the command line after /c, which command.com doesn't understand. But w9xpopen decodes the command line (in the runtime, before main() is called) and then reencodes it, this time quoting only arguments with spaces in them. Command.com then gets /c dir, and is happy. It would be interesting if this was the bug that led to w9xpopen being used in NT for the last ten years.

There are layers upon layers of brokenness here. w9xpopen should not be messing with the command line in the first place; it should call GetCommandLine() and pass the result untouched to CreateProcess (after skipping its own name). It certainly should not be using the argv[] contents, which are parsed with an algorithm that doesn't match the one used by cmd.exe. The decoding-encoding process munges the command line in hard-to-understand ways. Additionally, subprocess.py doesn't quote the shell name (my usual shell is C:\Program Files\TCCLE12\TCC.EXE), and it converts an argument list to a string using list2cmdline even when shell=True, which makes little sense to me.

I think w9xpopen should be deleted and forgotten. It was written badly and has apparently been largely ignored for 10+ years. There is probably a better solution to the problem even on Win9x, such as a worker thread in the Python process that waits on both the process and pipe handles. But also, all of the shell=True code in subprocess.py needs to be rethought from the ground up. I don't think it should exist at all; far better to provide convenient support in subprocess for setting up pipelines, and require people to explicitly invoke the shell for the few remaining legitimate use cases. That should probably be discussed elsewhere, though.
msg178644 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2012-12-31 06:45
This was fixed in #14470.
History
Date User Action Args
2012-12-31 06:45:57brian.curtinsetstatus: open -> closed

superseder: Remove using of w9xopen in subprocess module

nosy: + brian.curtin
messages: + msg178644
resolution: duplicate
stage: resolved
2011-03-08 04:36:37benrgsetnosy: loewis, amaury.forgeotdarc, techtonik, christian.heimes, trent, benrg
messages: + msg130305
2011-03-07 23:56:15benrgsetnosy: + benrg
messages: + msg130298
2011-02-20 06:27:56techtoniksetnosy: + techtonik

messages: + msg128894
versions: + Python 3.3
2010-11-14 21:20:47amaury.forgeotdarcsetfiles: + deprecate-w9xpopen.patch

nosy: + amaury.forgeotdarc
messages: + msg121208

keywords: + patch
2010-07-10 05:35:04terry.reedysetversions: + Python 3.2, - Python 2.6, Python 3.0
2008-03-21 18:21:19christian.heimessetpriority: normal
nosy: + christian.heimes
messages: + msg64249
components: + Windows
2008-03-18 22:51:00loewissetnosy: + loewis
messages: + msg64009
2008-03-18 20:09:28trentcreate