classification
Title: os.walk max_depth
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: andrekorol, loewis, palaviv, random832, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-04-16 14:11 by palaviv, last changed 2017-07-17 14:15 by rhettinger. This issue is now closed.

Files
File name Uploaded Description Edit
os-walk-max-depth.patch palaviv, 2016-04-16 14:11 review
os-walk-max-depth-2.patch palaviv, 2016-04-16 18:32 After SilentGhost review review
os.py andrekorol, 2017-07-17 13:25 os.py with lwalk function implemented
Repositories containing patches
https://github.com/andrekorol/cpython/
Messages (6)
msg263556 - (view) Author: Aviv Palivoda (palaviv) * Date: 2016-04-16 14:11
I am suggesting to add max_depth argument to os.walk. I think this is very useful for two cases. The trivial one is when someone wants to walk on a directory tree up to specific depth. The second one is when you follow symlinks and wish to avoid infinite loop.

The patch add the max_depth both to os.walk and os.fwalk.
msg263565 - (view) Author: (random832) Date: 2016-04-16 15:47
Wouldn't the "symlink infinite loop" case be better handled by making it track where it's already been? This can be done by inode and dev number on Unix; I'm not sure what equivalent exists on Windows (though symlinks are uncommon on Windows) but you could fall back on realpath.
msg263579 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-04-16 19:15
os.walk() allows more flexible control.

    for root, dirs, files in os.walk(top):
        if is_too_deep(root):
            dirs.clear()
            continue
        ...

You can even walk up to different depth on different parts of the tree. You can limit walking not by the directory depth, but by the length of the path, or the number of links in the path, or what-you-need. Adding separate parameters for all this particular cases is not practical.

I think there is a little need in this feature.
msg263861 - (view) Author: Aviv Palivoda (palaviv) * Date: 2016-04-20 20:35
I am not sure about the wide use of this feature but I do think this is a nice feature to add to os.walk. I can see how you can implement is_too_deep by counting the number of separators in the path. However I don't think it is trivial to do that. In addition doing the same when topdown is False will not work.
As for the other limitations you suggested I think they are less trivial then limiting the recursion depth.

As for the "symlink infinite loop" I agree that it will be better to do that by tracking the paths we have already visited but avoiding infinite loop is just a bonus that you get from this feature.
msg298510 - (view) Author: André Rossi Korol (andrekorol) * Date: 2017-07-17 13:25
I proposed a new function called lwalk(level walk) that recurses only to a certain level of depth: http://bugs.python.org/issue30942
It is implemented in os.py and calls os.walk, but making sure it recurses only to a selected level of depth.
If it is accepted I could send a Pull Request with the lwalk function implemented in os.py.
msg298524 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-07-17 14:15
> I think there is a little need in this feature.

I concur with Serhiy and think we're better-off without this proposal.
Marking this as closed.
History
Date User Action Args
2017-07-17 14:15:30rhettingersetstatus: open -> closed

nosy: + rhettinger
messages: + msg298524

resolution: rejected
stage: resolved
2017-07-17 13:25:48andrekorolsetfiles: + os.py

nosy: + andrekorol
messages: + msg298510

hgrepos: + hgrepo371
2017-07-17 12:40:03r.david.murraylinkissue30942 superseder
2016-04-20 20:35:03palavivsetmessages: + msg263861
2016-04-16 19:15:46serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg263579
2016-04-16 18:32:48palavivsetfiles: + os-walk-max-depth-2.patch
2016-04-16 15:47:51random832setnosy: + random832
messages: + msg263565
2016-04-16 14:11:10palavivcreate