This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: pathlib's iterdir doesn't specify what happens if directory content change
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, facundobatista, miss-islington, pitrou, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2020-05-03 16:36 by facundobatista, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22025 merged serhiy.storchaka, 2020-08-31 07:59
PR 22093 merged miss-islington, 2020-09-04 18:19
PR 22094 merged miss-islington, 2020-09-04 18:19
Messages (6)
msg367981 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2020-05-03 16:36
Documentation for Path.iterdir ( https://docs.python.org/3/library/pathlib.html#pathlib.Path.iterdir ) doesn't specify what happens when the directory change.

This is important, as the function does NOT return a list (which would imply that a "snapshot" of the directory will be returned).

Also, it's not only relevant what would happen if somebody else changes the directory after the iteration is started (will that change be reflected?) but also if it's ALLOWED for the same code that iterates the dir content to change it (or problems would arise, like when changing a dictionary while being iterated).
msg367987 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-05-03 17:25
There is no principal difference between Path.iterdir() and os.listdir(). Somebody can change the directory during iteration. The only difference is that in the latter case you iterate in C instead of Python, so it is less easy to catch a race condition. What would happen is OS and FS specific. Depending on OS and FS it may be even possible to get the same name several times when somebody quickly creates and removes entities in the directory.
msg376114 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2020-08-30 16:48
However, Serhiy, `os.listdir()` builds a list quickly and gives you that, so the chance of the directory being modified is quite low (however, for big directories, that may happen, and it's probably good to notice that in the docs).

For `Path.iterdir()` that list is not built, and as it's iteration is external, the amount of time between getting one item to the other is unbound, *anything* could happen in the middle. 

In short, I think that both docs should state that the directory could change while it's iterated (in the os.listdir case probably stating that the chances are low).
msg376393 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-09-04 18:19
New changeset 306cfb3a37e1438f6ba9f0a9f3af3c00aae4ec64 by Serhiy Storchaka in branch 'master':
bpo-40486: Specify what happens if directory content change diring iteration (GH-22025)
https://github.com/python/cpython/commit/306cfb3a37e1438f6ba9f0a9f3af3c00aae4ec64
msg376408 - (view) Author: miss-islington (miss-islington) Date: 2020-09-04 22:25
New changeset e52f5bc898c9a11fb1d57a42a1f9ec60b424d576 by Miss Islington (bot) in branch '3.8':
[3.8] bpo-40486: Specify what happens if directory content change diring iteration (GH-22025) (GH-22094)
https://github.com/python/cpython/commit/e52f5bc898c9a11fb1d57a42a1f9ec60b424d576
msg376410 - (view) Author: miss-islington (miss-islington) Date: 2020-09-04 22:29
New changeset bd078df28322f840afd363b6ba097b5506ea5098 by Miss Islington (bot) in branch '3.9':
[3.9] bpo-40486: Specify what happens if directory content change diring iteration (GH-22025) (GH-22093)
https://github.com/python/cpython/commit/bd078df28322f840afd363b6ba097b5506ea5098
History
Date User Action Args
2022-04-11 14:59:30adminsetgithub: 84666
2020-09-05 08:04:07serhiy.storchakasetstatus: open -> closed
type: enhancement
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.8, Python 3.10
2020-09-04 22:29:10miss-islingtonsetmessages: + msg376410
2020-09-04 22:25:57miss-islingtonsetmessages: + msg376408
2020-09-04 18:19:54miss-islingtonsetpull_requests: + pull_request21181
2020-09-04 18:19:46miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request21180
2020-09-04 18:19:37serhiy.storchakasetmessages: + msg376393
2020-08-31 07:59:19serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request21124
2020-08-30 16:48:49facundobatistasetmessages: + msg376114
2020-05-03 17:25:39serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg367987
2020-05-03 16:55:29xtreaksetnosy: + pitrou

title: pathlib's iterdir doesn't expecify what happens if directory content change -> pathlib's iterdir doesn't specify what happens if directory content change
2020-05-03 16:36:28facundobatistacreate