classification
Title: Race condition in distutils.dir_util.mkpath
Type: behavior Stage: resolved
Components: Distutils Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: Arfrever, djc, eric.araujo, tarek
Priority: normal Keywords: easy

Created on 2010-07-17 05:00 by Arfrever, last changed 2014-03-12 10:39 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
distutils-dir_util.patch Arfrever, 2010-07-17 05:00 Fix for distutils.dir_util.mkpath()
distutils2-mkpath.patch Arfrever, 2010-10-03 15:47 Fix for some mkpath functions in distutils2
Messages (11)
msg110538 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-07-17 05:00
distutils.dir_util.mkpath() is used to create a directory and is supposed to not fail when this directory already exists.
There's a chance that another process will create this directory between os.path.isdir() check and call to os.mkdir().
This was reproduced by a Gentoo user during parallel installation of Python:
https://bugs.gentoo.org/show_bug.cgi?id=311437

I'm attaching the patch.
msg110539 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-07-17 07:20
Note: Your patch fixes distutils, but in distutils2 dir_util is gone, use of its functions replaced by shutil calls or duplicated code with, I’m afraid, the same isdir/mkdir race condition. I wonder how we could add tests for that instead. Thanks for the heads-up, I’m adding a “grep makedirs|rmdir” on my todo list.
msg110540 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-07-17 07:21
instead of waiting for user reports*
msg117916 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-10-03 15:47
I'm attaching the patch for some mkpath functions in distutils2.
msg119703 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-10-27 12:14
Still, shouldn't this also be fixed in 2.7?
msg119707 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-27 12:57
Yes, it should, that’s why the versions field lists 2.7 too.

I’ll get to this in some days.
msg120582 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-06 04:54
Fixed in r86244 (py3k), r86245 (3.1), r86246 (2.7), thanks!

For distutils2, I’d like to commit a comprehensive fix.  Do you want to expand your patch to cover all instances of mkdir/makedirs?
msg122600 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-11-28 02:14
Maybe issue #9299 could be fixed first. Next you could backport new os.makedirs() in distutils2/_backport/os.py.
msg122606 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-28 02:39
We could backport the new version of the function in our compat module (I’d rather not backport the whole os module), if you think it’s worth it.
msg122607 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-11-28 02:43
I suggest to backport only makedirs() function.
msg124099 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-16 01:13
Do you want to backport the new makedirs function to d2/_backport/__init__.py?
History
Date User Action Args
2014-03-12 10:39:32eric.araujosetstatus: open -> closed
versions: + Python 3.1, Python 2.7, Python 3.2, - 3rd party
title: Race condition with mkdir/makedirs in distutils2 -> Race condition in distutils.dir_util.mkpath
components: + Distutils, - Distutils2
resolution: accepted -> fixed
stage: needs patch -> resolved
2010-12-16 01:13:06eric.araujosetkeywords: + easy, - patch
messages: + msg124099
nosy: tarek, djc, eric.araujo, Arfrever
stage: patch review -> needs patch
2010-11-28 02:43:03Arfreversetmessages: + msg122607
2010-11-28 02:39:31eric.araujosetmessages: + msg122606
2010-11-28 02:14:42Arfreversetmessages: + msg122600
2010-11-06 04:54:24eric.araujosetstatus: pending -> open
versions: - Python 3.1, Python 2.7, Python 3.2
nosy: tarek, djc, eric.araujo, Arfrever
title: Race condition in distutils.dir_util.mkpath() -> Race condition with mkdir/makedirs in distutils2
messages: + msg120582

components: - Distutils
2010-10-27 12:57:11eric.araujosetstatus: open -> pending
assignee: tarek -> eric.araujo
messages: + msg119707
2010-10-27 12:14:00djcsetnosy: + djc
messages: + msg119703
2010-10-03 15:47:03Arfreversetfiles: + distutils2-mkpath.patch
versions: + 3rd party, - Python 2.6
nosy: tarek, eric.araujo, Arfrever
messages: + msg117916

components: + Distutils2
2010-07-17 07:21:24eric.araujosetmessages: + msg110540
2010-07-17 07:20:22eric.araujosetnosy: + eric.araujo
messages: + msg110539
resolution: accepted

type: behavior
stage: patch review
2010-07-17 05:00:37Arfrevercreate