classification
Title: os.system() fails for commands with multiple quoted file names
Type: behavior Stage:
Components: Extension Modules, Windows Versions: Python 2.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Quigon, christian.heimes, d4rk1, georg.brandl, jafo, jfbastien, likes, zanella
Priority: normal Keywords:

Created on 2007-11-29 20:16 by Quigon, last changed 2008-07-23 16:47 by likes. This issue is now closed.

Files
File name Uploaded Description Edit
buggy.py Quigon, 2007-11-29 20:16
Messages (10)
msg57952 - (view) Author: Guy Mott (Quigon) Date: 2007-11-29 20:16
Given a call to os.system() with a command string like this:

   os.system('"TheCommand" > "MyOutput"')  # fails

then the call fails with the following error message on the console:

   'TheCommand" > "MyOutput' is not recognized as an internal or 
external command, operable program or batch file.

Note that both the executable file name and the redirected output file 
name are quoted.

Apparently the command string is being parsed and the first quote is 
incorrectly being matched with the last quote. A more general statement 
of this bug might say that multiple quoted fields on a command line 
cause os.system() to fail. I have not done enough research to better 
characterize the problem.

By contrast, if only one of the file names is quoted then the call to 
os.system() succeeds. E.g., these calls succeed:

   os.system('TheCommand > "MyOutput"')  # succeeds
   os.system('"TheCommand" > MyOutput')  # succeeds

Of course this is a simplified example where it is not necessary to 
quote either file name. Real world examples include 2 file names with 
imbedded spaces. E.g.:

   os.system('"The Command" > "My Output"')  # fails

   'The' is not recognized as an internal or external command, operable 
program or batch file.

A further real-world example is a command line with full path 
specifications for both the executable file and the output file. Such 
path specifications may include imbedded spaces so both need to be 
quoted. However, quoting both causes os.system() to fail. E.g.:

   os.system(r'"C:\New Folder\TheCommand" > "C:\New Folder\MyOutput"')  
# fails

   'C:\New' is not recognized as an internal or external command, 
operable program or batch file.

The above described scenario is the situation in the attached script 
that includes logic for finding an executable file that may not be 
found on the system path but is co-located with the Python script file. 
Thus the script and its companion file(s) may be moved from machine to 
machine and will work correctly even if not in a directory that is 
included on the system path. The script fails because the command line 
that it constructs, with executable and output file specifications 
quoted, fails in os.system().

Here is output from running the attached script:

-----------------------------------------------

C:\New Folder>buggy.py
strCmdLine=["ListMetadata" > "20071129Metadata.txt"]
'ListMetadata" > "20071129Metadata.txt' is not recognized as an 
internal or external command,
operable program or batch file.
Could not find "ListMetadata" on path, looking in script directory
strCmdLine=["C:\New Folder\ListMetadata" > "20071129Metadata.txt"]
'C:\New' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
  File "C:\New Folder\buggy.py", line 16, in <module>
    raise Exception("Could not locate command")
Exception: Could not locate command

-----------------------------------------------

Note that the command line that is constructed by the attached script 
runs just fine and produces the desired result if it is executed 
directly at a command line prompt. It is when executed via os.system() 
that the command line fails.

Testing environment:
   OS = Windows XP Professional
   Python = 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit 
(Intel)]
msg57960 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-29 22:32
I don't think that we can do anything about your problem. The user of
os.system() is responsible to do the quoting himself. os.system() is
just a tiny wrapper around the low level C function. We don't plan to
chance the fact that os.system() doesn't handling quoting.

However the subprocess module is clever enough to do the quoting for you.
msg57963 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-11-29 23:06
Are you sure that the exact command line works in a Windows shell?

Python does no processing on the string, it just hands it to the
platform system() function, so if MS decided that to work different from
a command prompt there's nothing we can do about.
msg57967 - (view) Author: Guy Mott (Quigon) Date: 2007-11-30 00:35
> Are you sure that the exact command line works in a Windows shell?

Yes, I tried running the exact same command line in a Windows shell and 
it worked fine. Note that the buggy.py script echoes the command line 
and then immediately calls os.system with it. E.g.:

   print "strCmdLine=[%s]" % strCmdLine
   nRtn = os.system(strCmdLine)

I tried running the script in a shell window, watched it fail, then 
immediately cut and pasted the command line that had been echoed out by 
the script to the command line prompt in the same shell, ran it and it 
succeeded.
msg62942 - (view) Author: Rafael Zanella (zanella) Date: 2008-02-24 21:39
I don't have access to a Windows machine, but is it really necessary to
quote the command part? I mean, on GNU/Linux if you pass a command wich
has spaces , say e.g.: ls -lah, quoted it fails too, but if passed
without quotes it runs just fine.
msg62981 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-02-25 13:13
Yes, it's necessary whenever the command part contains a space or other
special characters.
msg63298 - (view) Author: Jean-François Bastien (jfbastien) Date: 2008-03-05 22:18
I confirm the problem. To resolve it try:
os.system('call "TheCommand" > "MyOutput"')
msg64185 - (view) Author: Sean Reifschneider (jafo) * (Python committer) Date: 2008-03-20 17:53
I would agree with Georg that there isn't anything we can do about this.
 I had someone try from the Windows XP command shell and: "dir" "/w"
reports that it can't run the combined command, where: dir /w works just
fine.

My conclusions are:

This is a bug in the Windows shell (which os.system hands the command to).

There is a work-around using "call" as pointed out by Jean-François

The subprocess module is a better match for this, as you pass a tuple to
make quoting unnecessary.

According to the os module documentation, using subprocess is
recommended in preference to using os.system().
msg68717 - (view) Author: daniel.weyh (d4rk1) Date: 2008-06-25 07:57
Got similiar problem. Think it's a thing with the pipe '>'.

Try calling the windows-shell (e.g. C:\WINDOWS\system32\cmd.exe) with
'/C' and your comman dline after that (in quotes).
> subprocess.call(r'C:\WINDOWS\system32\cmd.exe /C "YourCommand >
YourOutput"')

For me it works when there are now newlines in YourCommand and YourOutput.
msg70177 - (view) Author: qiang (likes) Date: 2008-07-23 16:47
in subprocess.py , 
change line 788: args = comspec + " /c " + args
             to: args = comspec + args
it will be ok.
History
Date User Action Args
2008-07-23 16:47:53likessetnosy: + likes
messages: + msg70177
2008-06-25 07:57:39d4rk1setnosy: + d4rk1
messages: + msg68717
2008-03-20 17:53:46jafosetstatus: open -> closed
type: crash -> behavior
messages: + msg64185
priority: normal
nosy: + jafo
resolution: wont fix
2008-03-05 22:18:12jfbastiensetnosy: + jfbastien
messages: + msg63298
2008-02-25 13:13:24christian.heimessetmessages: + msg62981
2008-02-24 21:39:08zanellasetnosy: + zanella
messages: + msg62942
2007-11-30 00:35:54Quigonsetmessages: + msg57967
2007-11-29 23:06:26georg.brandlsetnosy: + georg.brandl
messages: + msg57963
2007-11-29 22:32:51christian.heimessetnosy: + christian.heimes
messages: + msg57960
2007-11-29 20:16:56Quigoncreate