Title: glob.glob incorrect results under windows when pathname exists but interpreter does not have access permissions to pathname
Type: behavior Stage: resolved
Components: Library (Lib), Windows Versions: Python 3.5
Status: closed Resolution: fixed
Dependencies: Superseder: os.stat fails when access is denied
View: 28075
Assigned To: Nosy List: eryksun, kwarzecha7, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2016-05-05 19:38 by kwarzecha7, last changed 2021-02-25 11:27 by eryksun. This issue is now closed.

Messages (5)
msg264917 - (view) Author: Krzysztof Warzecha (kwarzecha7) Date: 2016-05-05 19:38

I'm running python 3.5.1 under windows and I've noticed glob.glob is returning incorrect results when I'm using full path name to directory to which I don't have access permissions.

Please consider this:

    >> glob.glob('c:\\PerfLog*')
    >> glob.glob('c:\\PerfLogs')
    >> glob.glob('c:\\PerfLogs*')

I tried to replicate this under Linux with "chmod 000", but so far I'm unable to. One more example from Windows: "c:\\System Volume Information". This is also the case for user-created directories where python interpreter does not have access rights to directory - I have one like that on my client's machine.
msg264918 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-05 19:55
What returns os.lstat('c:\\PerfLogs')?
msg264919 - (view) Author: Krzysztof Warzecha (kwarzecha7) Date: 2016-05-05 19:58
>>> os.lstat('c:\\PerfLogs')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    PermissionError: [WinError 5] Odmowa dostępu: 'c:\\PerfLogs'

(thats "Permission denied")
msg264923 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-05-05 21:37
The access control list of a file may not grant or may deny the current user the right to read file attributes (i.e. lstat). Generally you don't have to worry about granted access to parent directories when attempting to stat the file or directory, since even standard users have SeChangeNotifyPrivilege to bypass path traversal access checks. In this case, ISTM that we at least know the file exists if lstat raises PermissionError instead of FileNotFoundError. Currently os.path.lexists returns False for any OSError, without distinguishing between these two cases.

You may also come across a similar issue when a file's delete disposition is set, since Windows doesn't allow opening a deleted file, even to read its attributes, and it doesn't actually unlink the file or directory until all references are closed. 

For example, create a directory, opened with delete sharing:

    >>> h = _winapi.CreateFile('testglob', 0x40000000, 7, 0, 1,
    ...                        0x2000000|0x1000000|0x80|0x10, 0)
    >>> os.path.isdir('testglob')

Before removing the directory, it can be globbed directly:

    >>> glob.glob('testglob')

After removing the directory, it can only be globbed indirectly by listing its parent directory:

    >>> os.rmdir('testglob')
    >>> os.lstat('testglob')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    PermissionError: [WinError 5] Access is denied: 'testglob'

    >>> glob.glob('testglob')
    >>> glob.glob('testglob*')

Once the handle is closed, the directory is unlinked:

    >>> _winapi.CloseHandle(h)
    >>> os.lstat('testglob')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    FileNotFoundError: [WinError 2] The system cannot find the file specified: 'testglob'
    >>> glob.glob('testglob*')
msg387658 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-02-25 11:27
Issue 28075 extended the os.stat() and os.lstat() implementation to query basic stat informatiom from the parent directory if opening the file fails with ERROR_ACCESS_DENIED. This change first became available in Python 3.5.3. 

Previously it was only querying the parent directory for the case of ERROR_SHARING_VIOLATION, which is unlikely since the open doesn't request and data access that requires sharing. It does handle one case that I know of. NTFS refuses to share even metadata access for a system paging file such as "C:\pagefile.sys".
Date User Action Args
2021-02-25 11:27:16eryksunsetstatus: open -> closed
superseder: os.stat fails when access is denied
messages: + msg387658

resolution: fixed
stage: resolved
2016-05-05 21:37:46eryksunsetnosy: + eryksun
messages: + msg264923
2016-05-05 19:58:53kwarzecha7setmessages: + msg264919
2016-05-05 19:55:32serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg264918
2016-05-05 19:38:31kwarzecha7create