classification
Title: Win32: shutil.copy leaks file handles to child processes
Type: security Stage:
Components: Library (Lib), Windows Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, akrpic77, haypo, hynek, r.david.murray, sbt, tarek
Priority: normal Keywords:

Created on 2013-04-04 16:23 by akrpic77, last changed 2014-07-30 21:39 by haypo. This issue is now closed.

Messages (6)
msg186041 - (view) Author: Andrej Krpic (akrpic77) Date: 2013-04-04 16:23
Function shutil.copy opens source file with open('rb') and destination file with open('wb') and then proceeds to copy the content.

If you create child process from the same python process during that time, that process will inherit open file handles.

This could lead to situation when parent process tries to delete the file, but child process prevents that by holding an open handle to the file (Windows XP, Windows 7).
 
This is not expected side effect for copy file operation.
Win32's native CopyFile API call doesn't leak file handles to child processes.

Python's open function behavior is defined by platform's implementation, which translates C function fopen with 'rb' and 'wb' modes to CRT open call without O_NOINHERIT flag, which creates inheritable Win32 HANDLEs.

Possible fix would be to add 'N' to mode string in calls to open function in shutil.copy.
msg186047 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-04-04 19:04
Unless I'm misunderstanding something, this has already been fixed in python3, where subprocess closes any open files (other than stdin/stdout/stderr) by default.  In Python2.7 (and possibly earlier, I haven't checked) you can get the same effect by specifying close_fds=True in the Popen call.

I don't see any reason why shutil should take any special action in this case, this is a general problem with open file descriptors and subprocess.
msg186072 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-04-05 10:11
This problem also affects processes started by multiprocessing (although it is fixed in http://hg.python.org/sandbox/sbt#spawn).

As far as I am concerned O_NOINHERIT should really have been applied by default in Python 3.  You only get inheritable fds on Windows if you use the msvcrt's spawn*() functions.  But in Python 3 os.spawn*() is implemented on top of subprocess so the fds are never inherited (even if the underlying handles are).
msg186074 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-04-05 11:59
See also the PEP 433 which proposes to disable file descriptor
inherance by default.

2013/4/5 Richard Oudkerk <report@bugs.python.org>:
>
> Richard Oudkerk added the comment:
>
> This problem also affects processes started by multiprocessing (although it is fixed in http://hg.python.org/sandbox/sbt#spawn).
>
> As far as I am concerned O_NOINHERIT should really have been applied by default in Python 3.  You only get inheritable fds on Windows if you use the msvcrt's spawn*() functions.  But in Python 3 os.spawn*() is implemented on top of subprocess so the fds are never inherited (even if the underlying handles are).
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue17634>
> _______________________________________
msg224344 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-30 21:35
PEP 446 supersedes PEP 433.
msg224346 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-07-30 21:39
The issue has been fixed in Python 3.4 with the PEP 446. I close this issue.

If you would like to enhance shutil.copyfile() by using CopyFile() on Windows, please open a new issue.
History
Date User Action Args
2014-07-30 21:39:54hayposetstatus: open -> closed
resolution: fixed
messages: + msg224346

versions: - Python 2.7, Python 3.5
2014-07-30 21:35:21BreamoreBoysetnosy: + BreamoreBoy

messages: + msg224344
versions: + Python 3.5, - Python 3.3
2013-04-05 11:59:25hayposetmessages: + msg186074
2013-04-05 10:11:26sbtsetmessages: + msg186072
2013-04-04 19:04:04r.david.murraysetnosy: + r.david.murray
messages: + msg186047
2013-04-04 18:24:24pitrousetnosy: + haypo, tarek, sbt, hynek

versions: + Python 3.4, - Python 2.6, Python 3.1, Python 3.2
2013-04-04 16:23:18akrpic77create