New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pathlib.Path().rglob() breaks with broken symlinks #80216
Comments
When using rglob() to iterate over the files of a directory containing a broken symlink (a link pointing to itself) rglob breaks with "[Errno 40] Too many levels of symbolic links" (OS: Linux). Steps to reproduce: mkdir tmp python3
>>> from pathlib import Path
>>> for f in Path().rglob("*"):
print(x)
foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/pathlib.py", line 1105, in rglob
for p in selector.select_from(self):
File "/usr/local/lib/python3.8/pathlib.py", line 552, in _select_from
for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
File "/usr/local/lib/python3.8/pathlib.py", line 536, in _iterate_directories
entry_is_dir = entry.is_dir()
OSError: [Errno 40] Too many levels of symbolic links: './foo'
What is more, stat(), is_dir(), is_file() and exists() also do not like those broken links:
>>> Path("foo").is_file()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/pathlib.py", line 1361, in is_file
return S_ISREG(self.stat().st_mode)
File "/usr/local/lib/python3.8/pathlib.py", line 1151, in stat
return self._accessor.stat(self)
OSError: [Errno 40] Too many levels of symbolic links: 'foo' Is this intended behaviour or is this a bug? I guess it's not intended, since it makes it impossible to iterate over such a directory with rglob(). I could not find anything similar in the bug tracker, but https://bugs.python.org/issue26012 seems to be related. Tested with Python 3.8.0a1, 3.6.7 and 3.5.2 (OS: Linux Mint 19) |
I confirm this issue with python 3.7 but your script is wrong (you declare f and use x in your script) /tmp$ mkdir demo /t/d/tmp$ python3 mar 19 fév 2019 13:20:15 CET
Python 3.7.2 (default, Jan 16 2019, 19:49:22)
[GCC 8.2.1 20181215 (Red Hat 8.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> for p in Path().rglob('*'):
... print(p)
...
foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.7/pathlib.py", line 1105, in rglob
for p in selector.select_from(self):
File "/usr/lib64/python3.7/pathlib.py", line 552, in _select_from
for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
File "/usr/lib64/python3.7/pathlib.py", line 536, in _iterate_directories
entry_is_dir = entry.is_dir()
OSError: [Errno 40] Too many levels of symbolic links: './foo'
>>> |
A possible solution for python 3.7+ could be to add "ELOOP" to _IGNORED_ERROS in pathlib but I'm not exactly sure of the side effects. |
3.5 is in security mode, we can remove 3.5 from the list for this issue. |
I consider this an enhancement since you do have a loop in your symlinks and so having it not exactly work isn't totally shocking. But that doesn't mean that if someone can come up with a reasonable solution to fixing this annoyance it woudn't be accepted or appreciated! |
And I don't know what a good solution would be. :) I.e. should some other exception be raised? Should it be ignored? I just don't know personally. |
In Windows, the error for a path reparse (e.g. symlink or junction) that can't be resolved is ERROR_CANT_RESOLVE_FILENAME. Another common error is ERROR_INVALID_REPARSE_DATA. This can occur if the reparse data is malformed or if the target device is invalid for the reparse type. For example, a junction is restricted to [bind-]mounting local volumes, so it's invalid if it targets a remote device. Windows errors to ignore should be added to _IGNORED_WINERRORS in Lib/pathlib.py. For example: _IGNORED_WINERRORS = (
# ...
1921, # ERROR_CANT_RESOLVE_FILENAME - similar to POSIX ELOOP
4392, # ERROR_INVALID_REPARSE_DATA - also for disallowed device targets
) |
Hello! I make the PR: #11964 But I need help to test it :-( Any could help me please? |
I tried to add a test file in #11988 I could only check on Linux and I have no idea how it behaves on any other OS. |
As expected the Windows CI build failed. Therefore, I added WinError 1921 to _IGNORED_WINERRORS as suggested by Eryk Sun. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: