classification
Title: Document that subprocess.Popen does not set PWD
Type: enhancement Stage:
Components: Documentation Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, jamesdlin, r.david.murray
Priority: normal Keywords:

Created on 2017-06-29 02:32 by jamesdlin, last changed 2017-07-02 00:38 by jamesdlin.

Messages (3)
msg297244 - (view) Author: James Lin (jamesdlin) Date: 2017-06-29 02:32
Even though http://bugs.python.org/issue4057 was rejected (which I think is fair), I think it would be worth mentioning something about PWD in the Python docs for subprocess.Popen's cwd parameter (and possibly for os.chdir):

1. It's pretty common for people to use Python to execute other things.
2. I don't think it's that uncommon to come across binaries/scripts that directly read PWD instead of calling getcwd() or equivalent.
3. For people who encounter this problem for the first time, it can be really puzzling.

My suggestion would be to add something like:

   Note: Some programs might expect that the current directory is specified in a PWD environment variable.  Callers should explicitly set it themselves when invoking such programs.

or even just:

   Note: Callers are responsible for setting a PWD environment variable if necessary.
msg297499 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-07-01 23:44
If you pass shell=True, PWD gests set, because it is a shell variable and the shell sets it.  If you don't, it doesn't, just like all the other shell-maintained variables.  I'm not sure it is worth calling out specifically in the Popen docs, myself; it is one consequence among *many* of calling a program directly rather than through a shell.  (Note that if a shell is involved at any point, PWD gets set, so it is only binary programs that require that they be called from a shell that are at issue.)

Furthermore, it is not obvious that even all shells set PWD, although I believe that most do.
msg297500 - (view) Author: James Lin (jamesdlin) Date: 2017-07-02 00:38
Yes, but the Python docs have scary-looking warnings about using shell=True, so people (rightly) should avoid using shell=True if they don't think that they need it.  And in this case, people might not even know that they're invoking some binary that expects PWD from the shell.  

In my experience, it's not that uncommon for people to rewrite sh/bash shell scripts in Python.  Something starts off as a sh/bash script, evolves to a point where it's cumbersome to maintain, and it gets rewritten in Python.  But after it's rewritten, some invoked binaries might not behave as before, and it's unclear why.

When I encountered this problem myself, the first thing I tried was to explicitly set the `cwd` argument.  Some mention of the PWD environment variable would have saved me a lot of time.
History
Date User Action Args
2017-07-02 00:38:27jamesdlinsetmessages: + msg297500
2017-07-01 23:44:27r.david.murraysetnosy: + r.david.murray
messages: + msg297499
2017-06-29 02:32:34jamesdlincreate