subprocess.py/list2cmdline should also put double quotes around strings that contain ampersands (&), but no spaces.
If not, the Windows command processor will split the command into two separate parts. In short, '&' needs the same treatment as '|'.

Thanks for reporting this issue. Can you attach a patch which adds a unit test covering this behavior and fixing the quoting rules? It would be very helpful in resolving this ticket. If implementing the whole thing is too much work, if you could just provide a failing unit test that would still be greatly helpful.

Thanks again.

Added patch file for subprocess.py and test_subprocess.py.

Thanks.

I'm not sure this is a correct change. And in fact, I would say that the current quoting of | is also incorrect.

& and | (and ^ and perhaps several others) have special meaning to cmd.exe. list2cmdline is documented as applying the quoting rules which the "MS C runtime" uses: cmd.exe and the C runtime are different and have different rules.

It seems to me that whoever added the | handling to list2cmdline was confused about the purpose of this function, or failed to properly document the function.

It would make more sense to document list2cmdline as applying cmd.exe-style quoting rules, if those are the rules it is actually going to implement.

A better option, though, would probably be to implement the cmd.exe quoting rules in a different function from the MS C runtime rules.

This all might benefit from a sanity check from someone who's actually worked with the subprocess module before, though (ie, not me).

http://www.autohotkey.net/~deleyd/parameters/parameters.htm#WINCRULES is a helpful reference, by the way.

A work-around could be that the caller puts double quotes around the individual elements of the sequence that need it.
However, this won't work because list2cmdline doesn't handle backslash quoting properly. An element like r'"foo"' is translated to r'\"foo\"'. This is incorrect because cmd.exe cannot handle this. The backslash may be appropriate for embedded quotes (like in r'foo"bar'), but not for outer quotes (like in r'"foobar"').

The user shouldn't have to worry with adding quotes anyway, so it would be better to demand that '|' and '&' are passed as separate elements in the sequence. Example ['echo', 'foo', '&', 'bar'].
When someone passes ['echo', 'foo&bar'], it is very obvious that r'echo "foo&bar"' is expected and not r'echo foo & bar'.

I have added a patch for this way of working of list2cmdline.

I'm not sure my last message was clear.

A work-around could be that ...

What problem is being worked around?

The discussion is going the wrong way. Let me state what is the actual problem.
When the Popen() function is called with "shell=True", you cannot pass command line elements in which the characters '&' and '|' are embedded (example r'Q&A'). list2cmdline should embed such elements in double quotes '"' in the same way as when a space is detected (so r'"Q&A"').
When '&' and '|' require their special meaning, they should be passed as separate elements (r'&') and in that case list2cmdline should not double-quote them.

The whole C++ discussion was only invoked because I said a potential work-around doesn't work because the way list2cmdline is designed. That remark is not relevant, because the actual problem is different.

That remark is not relevant, because the actual problem is different.

Maybe you can expand the test case to demonstrate the actual problem? The tests in the latest patch are for list2cmdline directly. But you can't observe the problem a bug in list2cmdline causes until you actually try to launch a child process. So perhaps the test should do that?

The actual bug here is that list2cmdline is called when shell=True and a list is passed. This should not be done. Instead, Popen should raise an TypeError (see bpo-7839).

When calling Popen with shell=True, you should be passing in a string that is already correctly quoted. This is how it works in Unix, and is how it should work in Windows as well.

So I agree with exarkun's comments in msg107679, and I think Gregory's fix in r60115 for bpo-1300 was incorrect and that in fact the OP was correct that he did not understand what was happening (note that he did not report an actual bug, he just reported that the code looked wrong to him).

I see your point about passing a command line as a single string instead of a list. Too bad that Popen doesn't just do the obvious thing (assemble the list into a string in a shell/cmd.exe compatible way).

bpo-1300 should indeed be reversed.

Raising ValueError will result in breaking some existing programs.

I've reverted the bpo-1300 revision from 2.6, 2.7, 3.1, and 3.2. I hope
7839 is resolved soon.

The reason that Popen doesn't do "the obvious thing" with a list and shell=True is that it is far from obvious what the correct thing is to do, especially for windows.

Thanks for the revert, exarkun.

bpo-11139 was closed as a duplicate of this issue.

I believe this is still valid in which case could we have the stage and resolution fields set appropriately please. You might also like to take a look at bpo-13238 which is referred to by bpo-7839.

The problem pointed to by this report was resolved by the revert. The issue of what to do about a list passed with shell=True is addressed (with no consensus) by bpo-7839. A proposal to add a windows equivalent of shlex.quote has been floated, but no takers have come forward to implement one, as far as I know.

There's nothing left to do here in this issue, as far as I can see.