classification
Title: Starting any program as a subprocess fails when subprocess.Popen has env argument
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.0, Python 3.1, Python 2.7, Python 2.6
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: georg.brandl Nosy List: georg.brandl, kermode, madprog, r.david.murray
Priority: normal Keywords: patch

Created on 2008-07-24 20:58 by kermode, last changed 2009-04-15 22:37 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
fail.py kermode, 2008-07-24 20:58 Python program that reproduces the bug
fail2.py madprog, 2009-04-09 15:54 Python program that reproduces the more general bug
issue3440-doc.patch r.david.murray, 2009-04-13 20:01 doc patch adding a warning about needing SystemRoot on Windows
Messages (9)
msg70225 - (view) Author: Lenard Lindstrom (kermode) Date: 2008-07-24 20:58
Python 2.6b2 (r26b2:65106, Jul 18 2008, 18:22:27) [MSC v.1500 32 bit
(Intel)] on win32

Windows XP Professional, SP 2

Library class subprocess.Popen


When subprocess.Popen is used to start the python interpreter as a
subprocess with a custom environment, env is set to a mapping, the
subprocess terminates with the following pop-up error dialog:

"The application failed to initialize properly (0xc0150004). Click on OK
to terminate the application."

The attached fail.py program reproduces the bug. It should print "ABCDE"
if successful. It does for Pythons 2.4 and 2.5.

The reason is that Python 2.6 on Windows requires the environment
variable SystemRoot, which it cannot find. This is demonstrated by
modifying fail.py to include SystemRoot in the environment. This works:

import sys
import os
import subprocess

command = ('''%s -c "import os; print os.environ['YAHOO']"''' %
           sys.executable)

env = {"YAHOO": "ABCDE", "SystemRoot": os.environ["SystemRoot"]}
subprocess.Popen(command, env=env).wait()
msg85818 - (view) Author: Paul Morelle (madprog) Date: 2009-04-09 15:54
Hello, I can reproduce the bug on a Windows XP Professional, SP 3, with
three versions of Python:

Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on
win32
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit
(Intel)] on win32
Python 2.6.1 (r261:67517, Dec  4 2008, 16:51:00) [MSC v.1500 32 bit
(Intel)] on win32

I can have this error code with the following code:

===================================================
import struct, subprocess

command = 'C:\\WINDOWS\\NOTEPAD.EXE'
env     = {'FOO': 'bar'}
p = subprocess.Popen(command, env=env)

p.wait()

err = struct.unpack('I', struct.pack('i', p.returncode))[0]
print '%x (%d)'%(err, err)
===================================================

The output is "c0150004 (3222601732)" with the three versions

If the `env' variable is set to None before the creation of the Popen
object, then Notepad.exe is launched correctly.

So, it is not only the python interpreter that fails with this call, it
would be the call to CreateProcess in the subprocess module that would
be incorrect…

Any hint?
msg85819 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-04-09 16:23
If you are passing a custom environment to a subprocess, then you must
supply everything in that environment that the program you are calling
requires in order to run.

That said, it would be a good idea to document the Windows minimum
environment requirements in the subprocess documentation.  Hopefully
someone can come up with a doc patch; as a non-Windows user I can't be
of much help with that.
msg85820 - (view) Author: Paul Morelle (madprog) Date: 2009-04-09 16:29
I have just figured out that if you initialize env with
os.environ.copy() and then add/modify its components, then the bug
disappears:

env = os.environ.copy()
env['FOO'] = 'BAR'
[…]

But I have no idea (for the moment) of which variable is mandatory or not.
msg85834 - (view) Author: Lenard Lindstrom (kermode) Date: 2009-04-09 22:57
The notepad example works with Pythons 2.4.4 and 2.5.4 on Windows 98. So
something changed in Windows XP. The 0xc0150004 error code crops up when
a side-by-side assembly fails to load. The DLL loader appears to use the
SystemRoot environment variable to find dependencies. I will check if
the XP notepad.exe is an SxS and post a follow-up.
msg85935 - (view) Author: Lenard Lindstrom (kermode) Date: 2009-04-13 04:03
notepad.exe forms a side-by-side assembly with COMCTL32.DLL. So
SystemRoot must be included in the environment. The following example
works with Python 2.5.2 on Windows XP.

===============================================================
import struct, subprocess
import os

command = 'C:\\WINDOWS\\NOTEPAD.EXE'
env     = {'FOO': 'bar', 'SystemRoot': os.environ['SystemRoot']}
p = subprocess.Popen(command, env=env)

p.wait()

err = struct.unpack('I', struct.pack('i', p.returncode))[0]
print '%x (%d)'%(err, err)
===============================================================

I would suggest Popen adds SystemRoot to env if it is not present.
msg85952 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-04-13 20:01
As I said, if you replace the environment it is your responsibility to
supply a working environment.  I've attached a doc patch that adds a
warning about this with the information that a valid SystemRoot is
needed to run a side-by-side assembly (whatever that is :).

If you think the doc patch is accurate I will commit it (unless Georg
objects).
msg85953 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-04-13 21:10
Looks good to me.
msg86003 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-04-15 22:37
Doc patch applied in r71631.
History
Date User Action Args
2009-04-15 22:37:40r.david.murraysetstatus: open -> closed
resolution: accepted
messages: + msg86003

stage: patch review -> resolved
2009-04-13 21:10:08georg.brandlsetmessages: + msg85953
2009-04-13 20:01:20r.david.murraysetfiles: + issue3440-doc.patch
keywords: + patch
messages: + msg85952

stage: needs patch -> patch review
2009-04-13 04:03:59kermodesetmessages: + msg85935
2009-04-09 22:57:30kermodesetmessages: + msg85834
2009-04-09 16:29:33madprogsetmessages: + msg85820
2009-04-09 16:23:41r.david.murraysetpriority: normal

type: crash -> enhancement
assignee: georg.brandl
components: + Documentation, - Library (Lib), Windows
versions: + Python 3.0, Python 3.1, Python 2.7
nosy: + georg.brandl, r.david.murray

messages: + msg85819
stage: needs patch
2009-04-09 15:54:18madprogsetfiles: + fail2.py
title: Starting Python as a subprocess fails when subprocess.Popen has env argument -> Starting any program as a subprocess fails when subprocess.Popen has env argument
nosy: + madprog

messages: + msg85818
2008-07-24 20:58:55kermodecreate