Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Windows] OSError when testing whether pathlib.Path('*') exists #79487

Closed
jimbo1qaz mannequin opened this issue Nov 24, 2018 · 8 comments
Closed

[Windows] OSError when testing whether pathlib.Path('*') exists #79487

jimbo1qaz mannequin opened this issue Nov 24, 2018 · 8 comments
Assignees
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes OS-windows stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@jimbo1qaz
Copy link
Mannequin

jimbo1qaz mannequin commented Nov 24, 2018

BPO 35306
Nosy @pfmoore, @pitrou, @tjguk, @zware, @serhiy-storchaka, @eryksun, @zooba, @miss-islington
PRs
  • bpo-35306: Handle '*' in pathlib.Path functions on Windows #11133
  • bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename #25529
  • [3.9] bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529) #25534
  • [3.8] bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529) #25535
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/zooba'
    closed_at = <Date 2021-04-22.20:12:38.885>
    created_at = <Date 2018-11-24.05:55:59.185>
    labels = ['type-bug', '3.8', '3.9', '3.10', 'library', 'OS-windows']
    title = "[Windows] OSError when testing whether pathlib.Path('*') exists"
    updated_at = <Date 2021-04-22.20:53:11.398>
    user = 'https://bugs.python.org/jimbo1qaz'

    bugs.python.org fields:

    activity = <Date 2021-04-22.20:53:11.398>
    actor = 'steve.dower'
    assignee = 'steve.dower'
    closed = True
    closed_date = <Date 2021-04-22.20:12:38.885>
    closer = 'steve.dower'
    components = ['Library (Lib)', 'Windows']
    creation = <Date 2018-11-24.05:55:59.185>
    creator = 'jimbo1qaz_'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 35306
    keywords = ['patch']
    message_count = 8.0
    messages = ['330371', '330394', '333092', '333728', '333730', '391627', '391630', '391631']
    nosy_count = 9.0
    nosy_names = ['paul.moore', 'pitrou', 'tim.golden', 'zach.ware', 'serhiy.storchaka', 'eryksun', 'steve.dower', 'miss-islington', 'jimbo1qaz_']
    pr_nums = ['11133', '25529', '25534', '25535']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue35306'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    @jimbo1qaz
    Copy link
    Mannequin Author

    jimbo1qaz mannequin commented Nov 24, 2018

    I'm writing a program taking paths from user input through CLI.

    path is a pathlib.Path().

    Since Windows doesn't expand asterisks, I check if the path doesn't exist. If so I expand using Path().glob(path).

    Unfortunately on Windows, if path (type: Path) contains asterisks, checking path.exists() or path.is_dir() raises WinError 123.

    Python 3.7.0 (default, Jun 28 2018, 08:04:48) [MSC v.1912 64 bit (AMD64)] :: Anaconda, Inc. on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from pathlib import Path
    >>> Path('*').exists()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1318, in exists
        self.stat()
      File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1140, in stat
        return self._accessor.stat(self)
    OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'
    >>> Path('*').is_dir()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1330, in is_dir
        return S_ISDIR(self.stat().st_mode)
      File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1140, in stat
        return self._accessor.stat(self)
    OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'

    I also reproduced on Miniconda 3.6.6, 3.7.0, and official Python 3.7.1.

    According to https://bugs.python.org/issue29827 , os.path.exists() (not Path.exists() ) returns False on any OSError.

    -----------------

    On Linux, checking paths with null bytes (a less common occurrence) raises a different error:

    >>> import pathlib
    >>> pathlib.Path("\x00").exists()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.6/pathlib.py", line 1336, in exists
        self.stat()
      File "/usr/lib/python3.6/pathlib.py", line 1158, in stat
        return self._accessor.stat(self)
      File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
        return strfunc(str(pathobj), *args)
    ValueError: embedded null byte

    @jimbo1qaz jimbo1qaz mannequin added 3.7 (EOL) end of life stdlib Python modules in the Lib dir labels Nov 24, 2018
    @eryksun
    Copy link
    Contributor

    eryksun commented Nov 25, 2018

    Path.exists should ignore all OSError exceptions, as os.path.exists does. Barring that, I suppose for Windows we could add EINVAL to IGNORED_ERROS.

    The problem with ValueError was already addressed in bpo-33721.

    @jimbo1qaz
    Copy link
    Mannequin Author

    jimbo1qaz mannequin commented Jan 6, 2019

    Should Path.resolve() also avoid raising OSError?

    Path('*').resolve()

    Traceback (most recent call last):
    ...truncated
      File "<ipython-input-5-4fa2fec5c8b3>", line 1, in <module>
        Path('*').resolve()
      File "C:\Users\jimbo1qaz\AppData\Local\Programs\Python\Python37\lib\pathlib.py", line 1134, in resolve
        s = self._flavour.resolve(self, strict=strict)
      File "C:\Users\jimbo1qaz\AppData\Local\Programs\Python\Python37\lib\pathlib.py", line 192, in resolve
        s = self._ext_to_normal(_getfinalpathname(s))
    OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'
    os.path.realpath('"*')
    Out[8]: 'C:\\Users\\jimbo1qaz\\Dropbox\\encrypted\\code\\corrscope\\"*'
    os.path.abspath('*"')
    Out[13]: 'C:\\Users\\jimbo1qaz\\Dropbox\\encrypted\\code\\corrscope\\*"'

    (sidenote: what os.path operation does Path.resolve() match? Path('nonexistent').resolve() returns a relative path on Python 3.7.1, whereas Path().resolve() returns an absolute path.)

    @zooba
    Copy link
    Member

    zooba commented Jan 15, 2019

    Pathlib doesn't necessarily directly follow os on its error handling - adding Antoine for comment.

    Passing strict=False to resolve() should be able to handle an invalid name like that. If not, I propose that we change it so that it does.

    @pitrou
    Copy link
    Member

    pitrou commented Jan 15, 2019

    I'm fine with swallowing the error in both exists() and resolve(). We should be careful not to swallow errors too broadly, though. The code paths should be audited to check that EINVAL can't mean something else.

    @pitrou pitrou added the 3.8 only security fixes label Jan 15, 2019
    @pitrou pitrou added the 3.9 only security fixes label Apr 17, 2020
    @eryksun eryksun added 3.10 only security fixes and removed 3.7 (EOL) end of life labels Mar 27, 2021
    @eryksun eryksun changed the title OSError [WinError 123] when testing if pathlib.Path('*') (asterisks) exists [Windows] OSError when testing whether pathlib.Path('*') exists Mar 27, 2021
    @eryksun eryksun added the type-bug An unexpected behavior, bug, or error label Mar 27, 2021
    @zooba
    Copy link
    Member

    zooba commented Apr 22, 2021

    New changeset 4696f12 by Steve Dower in branch 'master':
    bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
    4696f12

    @zooba zooba closed this as completed Apr 22, 2021
    @zooba zooba self-assigned this Apr 22, 2021
    @miss-islington
    Copy link
    Contributor

    New changeset 1575ea0 by Miss Islington (bot) in branch '3.8':
    bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
    1575ea0

    @miss-islington
    Copy link
    Contributor

    New changeset ab5d78c by Miss Islington (bot) in branch '3.9':
    bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
    ab5d78c

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes OS-windows stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants