classification
Title: [Windows] OSError when testing whether pathlib.Path('*') exists
Type: behavior Stage: resolved
Components: Library (Lib), Windows Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: eryksun, jimbo1qaz_, miss-islington, paul.moore, pitrou, serhiy.storchaka, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2018-11-24 05:55 by jimbo1qaz_, last changed 2021-04-22 20:53 by steve.dower. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 11133 closed v2m, 2018-12-12 20:15
PR 25529 merged steve.dower, 2021-04-22 19:41
PR 25534 merged miss-islington, 2021-04-22 20:04
PR 25535 merged miss-islington, 2021-04-22 20:05
Messages (8)
msg330371 - (view) Author: jimbo1qaz_ via Gmail (jimbo1qaz_) Date: 2018-11-24 05:55
I'm writing a program taking paths from user input through CLI.

`path` is a pathlib.Path().

Since Windows doesn't expand asterisks, I check if the path doesn't exist. If so I expand using Path().glob(path).

Unfortunately on Windows, if `path` (type: Path) contains asterisks, checking `path.exists()` or `path.is_dir()` raises WinError 123.

Python 3.7.0 (default, Jun 28 2018, 08:04:48) [MSC v.1912 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> Path('*').exists()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1318, in exists
    self.stat()
  File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1140, in stat
    return self._accessor.stat(self)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'
>>> Path('*').is_dir()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1330, in is_dir
    return S_ISDIR(self.stat().st_mode)
  File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1140, in stat
    return self._accessor.stat(self)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'

I also reproduced on Miniconda 3.6.6, 3.7.0, and official Python 3.7.1.

According to https://bugs.python.org/issue29827 , os.path.exists() (not Path.exists() ) returns False on any OSError.

-----------------

On Linux, checking paths with null bytes (a less common occurrence) raises a different error:

>>> import pathlib
>>> pathlib.Path("\x00").exists()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/pathlib.py", line 1336, in exists
    self.stat()
  File "/usr/lib/python3.6/pathlib.py", line 1158, in stat
    return self._accessor.stat(self)
  File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
ValueError: embedded null byte
msg330394 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2018-11-25 20:32
Path.exists should ignore all OSError exceptions, as os.path.exists does. Barring that, I suppose for Windows we could add EINVAL to IGNORED_ERROS. 

The problem with ValueError was already addressed in issue 33721.
msg333092 - (view) Author: jimbo1qaz_ via Gmail (jimbo1qaz_) Date: 2019-01-06 08:10
Should Path.resolve() also avoid raising OSError?

Path('*').resolve()

Traceback (most recent call last):
...truncated
  File "<ipython-input-5-4fa2fec5c8b3>", line 1, in <module>
    Path('*').resolve()
  File "C:\Users\jimbo1qaz\AppData\Local\Programs\Python\Python37\lib\pathlib.py", line 1134, in resolve
    s = self._flavour.resolve(self, strict=strict)
  File "C:\Users\jimbo1qaz\AppData\Local\Programs\Python\Python37\lib\pathlib.py", line 192, in resolve
    s = self._ext_to_normal(_getfinalpathname(s))
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'


os.path.realpath('"*')
Out[8]: 'C:\\Users\\jimbo1qaz\\Dropbox\\encrypted\\code\\corrscope\\"*'
os.path.abspath('*"')
Out[13]: 'C:\\Users\\jimbo1qaz\\Dropbox\\encrypted\\code\\corrscope\\*"'

(sidenote: what os.path operation does Path.resolve() match? Path('nonexistent').resolve() returns a relative path on Python 3.7.1, whereas Path().resolve() returns an absolute path.)
msg333728 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-01-15 18:40
Pathlib doesn't necessarily directly follow os on its error handling - adding Antoine for comment.

Passing strict=False to resolve() should be able to handle an invalid name like that. If not, I propose that we change it so that it does.
msg333730 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2019-01-15 19:02
I'm fine with swallowing the error in both exists() and resolve(). We should be careful not to swallow errors too broadly, though.  The code paths should be audited to check that EINVAL can't mean something else.
msg391627 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-04-22 20:04
New changeset 4696f1285d83dd7b69f459c63e14080b1f87f797 by Steve Dower in branch 'master':
bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
https://github.com/python/cpython/commit/4696f1285d83dd7b69f459c63e14080b1f87f797
msg391630 - (view) Author: miss-islington (miss-islington) Date: 2021-04-22 20:25
New changeset 1575ea01059b3ba836cdb62f01f0b6584d7756a7 by Miss Islington (bot) in branch '3.8':
bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
https://github.com/python/cpython/commit/1575ea01059b3ba836cdb62f01f0b6584d7756a7
msg391631 - (view) Author: miss-islington (miss-islington) Date: 2021-04-22 20:30
New changeset ab5d78ce4141a6b3cd55588cdbe3cd3bd5242d5b by Miss Islington (bot) in branch '3.9':
bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
https://github.com/python/cpython/commit/ab5d78ce4141a6b3cd55588cdbe3cd3bd5242d5b
History
Date User Action Args
2021-04-22 20:53:11steve.dowersetpull_requests: - pull_request24257
2021-04-22 20:51:44steve.dowersetpull_requests: + pull_request24257
2021-04-22 20:30:43miss-islingtonsetmessages: + msg391631
2021-04-22 20:25:17miss-islingtonsetmessages: + msg391630
2021-04-22 20:12:38steve.dowersetstatus: open -> closed
assignee: steve.dower
resolution: fixed
stage: patch review -> resolved
2021-04-22 20:05:01miss-islingtonsetpull_requests: + pull_request24254
2021-04-22 20:04:54miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request24253
2021-04-22 20:04:47steve.dowersetmessages: + msg391627
2021-04-22 19:41:27steve.dowersetpull_requests: + pull_request24248
2021-03-27 05:12:40eryksunlinkissue42855 superseder
2021-03-27 05:12:15eryksunsettitle: OSError [WinError 123] when testing if pathlib.Path('*') (asterisks) exists -> [Windows] OSError when testing whether pathlib.Path('*') exists
type: behavior
versions: + Python 3.10, - Python 3.7
2021-03-27 05:08:43eryksunsetmessages: - msg333746
2020-04-17 18:04:13pitrousetversions: + Python 3.9
2019-01-16 01:14:21eryksunsetmessages: + msg333746
2019-01-15 19:02:30pitrousetmessages: + msg333730
versions: + Python 3.8, - Python 3.6
2019-01-15 18:40:28steve.dowersetnosy: + pitrou
messages: + msg333728
2019-01-06 08:10:56jimbo1qaz_setmessages: + msg333092
2018-12-12 20:15:52v2msetkeywords: + patch
stage: patch review
pull_requests: + pull_request10365
2018-11-25 20:32:24eryksunsetnosy: + eryksun, paul.moore, tim.golden, serhiy.storchaka, zach.ware, steve.dower
messages: + msg330394
components: + Windows
2018-11-24 05:55:59jimbo1qaz_create