classification
Title: shutil.copyfile blocks indefinitely on named pipes
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.0, Python 2.6, Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: aioryi, draghuram, farialima, pitrou, schmir
Priority: critical Keywords: patch

Created on 2008-05-29 15:32 by aioryi, last changed 2009-05-01 21:10 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
issue3002.patch pitrou, 2009-03-08 23:23
issue3002-2.patch pitrou, 2009-03-08 23:30
Messages (10)
msg67498 - (view) Author: albert hofkamp (aioryi) Date: 2008-05-29 15:32
shutil.copytree() uses shutil.copyfile() to copy files recursively.
shutil.copyfile() opens the source file for reading, and the destination
file for writing, followed by a call to shutil.copyfileobj().

If the file happens to be a named pipe rather than a normal file,
opening for read blocks the copying function, since the Unix OS needs a
writer process to attach to the same named pipe before the open-for-read
succeeds.

Rather than opening the file for reading, the correct action would
probably be to simply create a new named pipe with the same name at the
destination.
Looking at the Python2.3 code, the same type of problem seem to exist
for other non-normal file-types other than symlinks (eg device files,
sockets, and possibly a few others).
msg67499 - (view) Author: Raghuram Devarakonda (draghuram) Date: 2008-05-29 15:45
I am not sure if copyfile() should be trying to copy named pipes (or any
other special files for that matter). The best way is perhaps to check
and skip such files.
msg67700 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-04 20:48
I can confirm this issue on python 2.5.
I think the priority should be set to critical, as this can lead to
denial of service attacks.
msg67758 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-06 09:43
The open('fifo', 'rb') already blocks. One has to use os.fdopen with
O_NONBLOCK to prevent blocking.

fd=os.open('fifo', os.O_RDONLY | os.O_NONBLOCK)

and then use

stat.S_ISFIFO(os.fstat(fd).st_mode)

to check if this is a fifo.
Checking with os.stat before opening with open will result in a race
condition.
msg67773 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-06 14:57
if the destination is a named pipe, it will also block (waiting for a
reader).
msg81802 - (view) Author: Fran├žois Granade (farialima) Date: 2009-02-12 18:49
Note that the 'cp' user command in Linux blocks also. Although this is
*very* painful, you could argue that since the OS commands do the same
thing, it shouldn't be fixed.
msg83312 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-08 14:39
I'm not sure the race condition possibility is important. Is
shutil.copytree() protected against such race conditions in the first place?

(I'd argue this bug is less about potential DOS attacks than the simple
unfriendliness of indefinitely blocking)
msg83335 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-08 23:23
Here is a patch, introducing a new exception named SpecialFileError
which is raised when trying to copy a named pipe. Other kinds of special
file aren't checked for, but they could it we thought it's sensible to
do so.
msg83336 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-08 23:30
I forgot the case where the /destination/ file is a named pipe. Here is
a new patch.
msg86910 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-05-01 21:10
Committed in r72178, r72179. Thanks!
History
Date User Action Args
2009-05-01 21:10:30pitrousetstatus: open -> closed
resolution: fixed
messages: + msg86910
2009-03-08 23:30:57pitrousetfiles: + issue3002-2.patch

messages: + msg83336
2009-03-08 23:23:51pitrousetfiles: + issue3002.patch

messages: + msg83335
keywords: + patch
2009-03-08 14:39:59pitrousetnosy: + pitrou
messages: + msg83312
2009-02-12 18:49:59farialimasetnosy: + farialima
messages: + msg81802
2008-09-04 01:18:23benjamin.petersonsetpriority: release blocker -> critical
2008-08-21 14:56:11benjamin.petersonsetpriority: critical -> release blocker
2008-06-06 14:57:48schmirsetmessages: + msg67773
2008-06-06 09:43:17schmirsetmessages: + msg67758
2008-06-06 09:10:47georg.brandlsetpriority: critical
versions: + Python 3.0, - Python 2.4, Python 2.3
2008-06-04 20:49:00schmirsetnosy: + schmir
messages: + msg67700
versions: + Python 2.6, Python 2.5, Python 2.4
2008-05-29 15:45:46draghuramsetnosy: + draghuram
messages: + msg67499
2008-05-29 15:32:29aioryicreate