classification
Title: Restore os.makedirs ability to apply mode to all directories created
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: JustAnotherArchivist, ZackerySpytz, awtimmering, christian.heimes, gregory.p.smith, kartiksubbarao, serhiy.storchaka
Priority: normal Keywords: 3.7regression, patch

Created on 2020-11-16 09:54 by gregory.p.smith, last changed 2021-11-22 21:16 by gregory.p.smith.

Pull Requests
URL Status Linked Edit
PR 23901 open ZackerySpytz, 2020-12-22 23:58
Messages (5)
msg381082 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2020-11-16 09:54
os.makedirs used to pass its mode argument on down recursively so that every level of directory it created would have the specified permissions.

That was removed in Python 3.7 as https://bugs.python.org/issue19930 as if it were a mis-feature.  Maybe it was in one situation, but it was also a desirable *feature*.  It wasn't a bug.

We've got 15 year old code depending on that and the only solution for it to work on Python 3.7-3.9 is to reimplement recursive directory creation. :(

This feature needs to be brought back.  Rather than flip flop on the API, adding an flag to indicate if the permissions should be applied recursively or not seems like the best way forward.

The "workaround" documented in the above bug is invalid.  umask cannot be used to control the intermediate directory creation permissions as that is a process wide global that would impact other threads or signal handlers.  umask also cannot be used as umask does not allow setting of special bits such as stat.S_ISVTX.

result: Our old os.makedirs() code that tried to set the sticky bit (ISVTX) on all directories now fails to set it at all levels.
msg381084 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2020-11-16 10:01
For consistency, pathlib.Path.mkdir should also gain this option.
msg381086 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-11-16 10:32
Alternatively we can add an optional argument for the mode of intermediate directories.

makedirs(name, mode=0o777, exist_ok=False, intermediate_mode=0o777)
msg381087 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-11-16 10:36
+1 for restoring the feature

+0 for Serhiy's proposal iff intermediate_mode defaults to the value of mode:


    def makedirs(name, mode=0o777, exist_ok=False, *, intermediate_mode=None):
        if intermediate_mode is None:
            intermediate_mode = mode
msg406802 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2021-11-22 21:16
A new `intermediate_mode=` kwarg can't _default_ to `mode=` as that would flip flop the API's default behavior again and no doubt disrupt someone elses 3.7-3.10 authored code depending on it. :(

Regardless I do somewhat like `intermediate_mode=` more than the PR's existing `recursive_mode=` as it opens up more possible behaviors... Allowing `None` to mean "use `mode=`" is desirable as that'll likely be the most common case.  As I expect this to be very common, an easier to type name than intermediate_mode is desirable.  Brainstorming:

 * mid_mode?
 * submode?
 * imode?

My preference leans towards the latter names. Probably `submode=`.
History
Date User Action Args
2021-11-22 21:16:30gregory.p.smithsetmessages: + msg406802
2021-11-19 12:13:32awtimmeringsetnosy: + awtimmering
2021-10-26 18:29:17gregory.p.smithsetversions: + Python 3.11, - Python 3.10
2021-10-26 15:39:29kartiksubbaraosetnosy: + kartiksubbarao
2021-05-05 01:22:10JustAnotherArchivistsetnosy: + JustAnotherArchivist
2020-12-22 23:58:03ZackerySpytzsetkeywords: + patch
nosy: + ZackerySpytz

pull_requests: + pull_request22756
stage: needs patch -> patch review
2020-11-16 10:36:36christian.heimessetnosy: + christian.heimes
messages: + msg381087
2020-11-16 10:32:46serhiy.storchakasetmessages: + msg381086
2020-11-16 10:01:31gregory.p.smithsettype: behavior -> enhancement
2020-11-16 10:01:17gregory.p.smithsetmessages: + msg381084
2020-11-16 09:54:11gregory.p.smithcreate