Title: test_popen fails on Windows if installed to "Program Files"
Type: behavior Stage: needs patch
Components: Library (Lib), Windows Versions: Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: tim.golden Nosy List: BreamoreBoy, JosephArmbruster, eryksun, ezio.melotti, flox, georg.brandl, loewis, pjenvey, squeegee, steve.dower, tim.golden
Priority: normal Keywords: buildbot, needs review, patch

Created on 2006-09-15 12:59 by loewis, last changed 2015-02-19 02:16 by steve.dower.

File name Uploaded Description Edit
popen.diff loewis, 2006-09-15 12:59
issue1559298_trunk.diff brian.curtin, 2010-01-14 22:09 patch and docs against r77505
issue1559298_py3k.diff brian.curtin, 2010-01-14 22:12 py3k version of the patch
Messages (14)
msg51121 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-09-15 12:59
test_popen fails in 2.5c2. The reason is that popen invokes

cmd.exe /c "c:\program files\python25\python.exe" -c
"import sys;print sys.argv"

cmd.exe does not support that syntax, and gives an
error (which silently disappears); the pipe read then
returns an empty string.

This problem exists atleast since Python 2.3.

To fix this, cmd.exe needs to be invoked as

cmd.exe /c "c:\program files\python25\python.exe" -c
"import sys;print sys.argv"

The attached patch fixes this by always wrapping the
command line with an addition pair of quotes.

It's not clear to me whether this can go into 2.5.1: an
application may already work around this problem by
passing extra quotes to popen, which would then break
if popen adds even more quotes.
msg51122 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-03-13 09:39
I don't see a difference between your two command lines here... did you mean to add additional quotes in the second example?
msg51123 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-03-14 19:50
Indeed, this should read

cmd.exe /c ""c:\program files\python25\python.exe" -c "import sys;print sys.argv""
msg57615 - (view) Author: Joseph Armbruster (JosephArmbruster) Date: 2007-11-18 13:54
I applied the change to:

Python 2.6a0 (trunk:58651M, Nov 18 2007, 08:46:54) [MSC v.1400 32 bit
(Intel)] on win32

and test_popen passes appeared to pass.
msg88328 - (view) Author: Philip Jenvey (pjenvey) * (Python committer) Date: 2009-05-26 02:25
subprocess also needs this fix applied

Does the w9xopen command line below not need this?
msg90104 - (view) Author: Russ Gibson (squeegee) Date: 2009-07-04 04:14
What is needed is separate quoting for the command and the argument. 
Right now, test_popen only surrounds the entire command line with quotes:

"c:\Program Files\Python2.6\Python.exe -u c:\Documents and Settings\Russ

It needs to the above as thus:

"c:\Program Files\Python2.6\Python.exe" -u "c:\Documents and
Settings\Russ Gibson\cgi-bin\"

From the command line, the second case works, but the first doesn't. 
Neither work from os.popen in 2.6.2.

To make sure that popen works in all cases under windows, test_popen*
need to test with separate quotes around the command and argument.  That
will show the real problem with the current os.popen* implementation(s).

(yeah, I know, shut up and provide a patch...)
msg97790 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-01-14 22:09
This has come up recently and Martin's approach seems to work. I updated his patch for trunk, and test_popen passes when I run it with and without a space in the path. 

Russ Gibson's suggestion was already applied to test_popen, so the only code change is to posixmodule.c.
msg97791 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-01-14 22:12
Here is a py3k version of the previous patch. os.popen is implemented using subprocess.Popen, so the quoting change was made there.
msg97802 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-01-15 08:52
It is on Python 2.7a2

>>> os.popen('"C:\\Python27\\python.exe" -c "import sys; print sys.argv" 42').read()

>>> os.popen('""C:\\Python27\\python.exe" -c "import sys; print sys.argv" 42"').read()
"['-c', '42']\n"

>>> os.popen3('"C:\\Python27\\python.exe" -c "import sys; print sys.argv" 42')[2].read()
'\'C:\\Python27\\python.exe" -c "import\' est pas reconnu en tant que commande interne ou externe, un programme executable ou un fichier de commandes.\n'

It may be related to the test_popen failure.
Actually, it seems that the outer double quotes are removed before executing the cmdstring:
  C:\Python27\python.exe" -c "import
 ^                                  ^
msg98512 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-01-29 14:25
Florent is correct. The patch seems to fix regular popen, but popen3 sees problems. I'll see if I can fit this in and have a look.

Also of note is that the other flavors of popen are not least not in Lib/test/ or Lib/test/
msg219999 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-07 23:35
Is this worth pursuing given the wording here ?
msg220020 - (view) Author: Eryk Sun (eryksun) * Date: 2014-06-08 06:36
This is fixed for subprocess.Popen in 2.7, 3.1, and 3.2; see issue 2304. In 2.7, nt.popen still has this problem. As mentioned above, it can be worked around by using subprocess.Popen as described here:
msg236175 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015-02-18 14:55
Do we leave this open or close it as there is a known work around for 2.7?
msg236209 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-02-19 02:16
The docs are pretty clear about using Popen instead of os.popen, and people using popen have likely worked around it already, so we're more likely to break them by changing it now.

I vote to close.
Date User Action Args
2015-02-19 02:16:23steve.dowersetmessages: + msg236209
2015-02-18 16:35:52ezio.melottisetnosy: + steve.dower
2015-02-18 14:55:15BreamoreBoysetmessages: + msg236175
versions: - Python 3.1, Python 3.2
2014-06-08 06:36:45eryksunsetnosy: + eryksun
messages: + msg220020
2014-06-08 00:17:37brian.curtinsetnosy: - brian.curtin
2014-06-07 23:35:44BreamoreBoysetnosy: + BreamoreBoy
messages: + msg219999
2010-08-25 07:39:39tim.goldensetassignee: tim.golden

nosy: + tim.golden
2010-08-24 20:33:43BreamoreBoysetversions: - Python 2.6
2010-01-29 14:25:07brian.curtinsetmessages: + msg98512
2010-01-29 13:11:01ezio.melottisetstage: patch review -> needs patch
2010-01-26 17:34:28ezio.melottisetnosy: + ezio.melotti
2010-01-15 16:26:30floxsetkeywords: + buildbot
2010-01-15 08:52:20floxsetnosy: + flox
messages: + msg97802
2010-01-14 22:12:04brian.curtinsetfiles: + issue1559298_py3k.diff

messages: + msg97791
2010-01-14 22:09:35brian.curtinsetfiles: + issue1559298_trunk.diff

components: + Library (Lib), - Tests
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 3.0
keywords: + needs review
nosy: + brian.curtin

messages: + msg97790
2009-07-04 04:14:47squeegeesetnosy: + squeegee
messages: + msg90104
2009-05-26 02:25:11pjenveysetnosy: + pjenvey
messages: + msg88328
2009-03-30 03:08:38ajaksu2setstage: patch review
type: behavior
components: + Tests
versions: + Python 2.6, Python 3.0
2007-11-18 13:54:58JosephArmbrustersetnosy: + JosephArmbruster
messages: + msg57615
2006-09-15 12:59:09loewiscreate