This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: [doc] os.walk is limited by python's recursion limit
Type: Stage: resolved
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Elena.Oat, Sergey Chvalyuk, Thomas.Waldmann, docs@python, iritkatriel, rhettinger, rmast, slateny
Priority: low Keywords: easy

Created on 2016-03-12 00:59 by Thomas.Waldmann, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (7)
msg261626 - (view) Author: Thomas Waldmann (Thomas.Waldmann) Date: 2016-03-12 00:58
os.walk calls itself recursively and that limits the directory depth it can "walk" into.

On Linux, one can create directory hierarchies deeper than that.

For some more details see there:
msg261627 - (view) Author: Thomas Waldmann (Thomas.Waldmann) Date: 2016-03-12 01:07
Note: similar issues can be seen in other stdlib recursive filesystem-related functions also.
msg261681 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-03-13 07:22
I think the code should be left as-is (we've used some variation of recursion for walking a very long time with no reported real-world problems).  Perhaps a documentation note can be added to the effect that the there is a recursion limit and that it can be changed by the user if needed.
msg262064 - (view) Author: Elena Oat (Elena.Oat) * Date: 2016-03-20 10:07
I've actually tried to test the recursion limit on my Mac OS X. It seems that I cannot create any subdirectories after around 500 subdir depth. I guess it's related to the allowed path length (<1024 chars?).
msg268116 - (view) Author: Sergey Chvalyuk (Sergey Chvalyuk) Date: 2016-06-10 11:56
There is another limitation with os.walk
it also cannot go very deep inside tree because of scandir can throw 
OSError(36, 'File name too long')

I have wrote tests for both cases:

tested on linux only

on MacOS it throw 'File name too long' for 1-st test too
set sys.settrecursionlimit to lower values
msg414334 - (view) Author: Stanley (slateny) * Date: 2022-03-02 09:43
I'm not too sure about documenting the recursive limit here. There's a few other recursive functions in the os library (such as makedirs()) and if we note the recursive limit for os.walk then all of them should be noted too, but that doesn't seem quite right to me.
msg415330 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-03-16 09:17
I agree with Stanley. The documentation for os is clear that recursion is used and the documentation for RecursionError links to getrecursionlimit(). This seems sufficient.
Date User Action Args
2022-04-11 14:58:28adminsetgithub: 70732
2022-03-16 09:17:27iritkatrielsetstatus: open -> closed

nosy: + iritkatriel
messages: + msg415330

resolution: wont fix
stage: resolved
2022-03-02 09:43:54slatenysetnosy: + slateny
messages: + msg414334
2021-08-18 06:30:22rmastsetnosy: + rmast
2021-06-22 13:42:59iritkatrielsetkeywords: + easy
title: os.walk is limited by python's recursion limit -> [doc] os.walk is limited by python's recursion limit
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.5
2016-06-10 11:56:12Sergey Chvalyuksetnosy: + Sergey Chvalyuk
messages: + msg268116
2016-03-20 10:07:38Elena.Oatsetnosy: + Elena.Oat
messages: + msg262064
2016-03-13 07:22:35rhettingersetpriority: normal -> low

nosy: + rhettinger, docs@python
messages: + msg261681

assignee: docs@python
components: + Documentation, - Library (Lib)
2016-03-12 01:07:20Thomas.Waldmannsetmessages: + msg261627
2016-03-12 00:59:00Thomas.Waldmanncreate