classification
Title: Improve error messages for Windows reserved file names
Type: enhancement Stage:
Components: Windows Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: CarK, SilentGhost, eryksun, paul.moore, steve.dower, steven.daprano, tim.golden, zach.ware
Priority: normal Keywords: 3.6regression

Created on 2019-07-07 00:00 by steven.daprano, last changed 2019-07-10 08:38 by eryksun.

Messages (4)
msg347456 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2019-07-07 00:00
See #37515.

Perhaps Windows builds can check for reserved file names and give a more descriptive error message in the event of IO error?

(Eryksun also mentions two reserved names which Microsoft apparently does not document: "CONIN$" and "CONOUT$".)
msg347465 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-07-07 04:03
> Perhaps Windows builds can check for reserved file names and give a
> more descriptive error message in the event of IO error?

An operation on a reserved DOS device name can also succeed with unexpected results. For example, a script may unintentionally write to the active console screen buffer, "conout$":

    >>> open('C:/conout$::. .::.dat', 'w').write('spam\n')
    spam
    5 

There's also the issue of normalization that removes trailing spaces and dots from the final path component. All paths get normalized, except for device paths that begin with exactly "\\?\" (i.e. extended paths) in a create or open context. For example, say a script creates a file with the reserved name "spam. . .":

    >>> open(r'\\?\C:\Temp\spam. . .', 'w').close()

Then later, it generically uses os.walk('C:/Temp'), without the "\\?\" prefix, and tries to remove the file:

    >>> os.remove('C:/Temp/spam. . .')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:/Temp/spam. . .'

Without an extended path, "spam. . ." gets normalized as "spam". The script would need to use os.walk(r'\\?\C:\Temp'). Should we special case this error as well to suggest using an extended path?

> Eryksun also mentions two reserved names which Microsoft apparently 
> does not document: "CONIN$" and "CONOUT$".

The system's behavior with these two names depends on the Windows version. In Windows 7 and earlier, "CONIN$" and "CONOUT$" are special cased by CreateFileW, and only when it's just the bare names (case insensitive) without trailing colons, spaces, or an extension, and never in a directory. In Windows 8+, as part of updating the internal console implementation to use an I/O device (i.e. "\Device\ConDrv"), "CONIN$" and "CONOUT$" were added to the system runtime library's list of DOS devices, so they behave the same as other DOS device names, including "NUL", "CON", "AUX", "PRN", "COM<1-9>", and "LPT<1-9>". This change is undocumented.
msg347537 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-07-09 10:11
Presumably this is linked to issue37515 (why not just repurpose that one?), but I'm inclined to think this is okay provided:

* if the operation succeeds, no error/message is displayed
* if the operation fails, we only update the error message if one of the special filenames is the target

So probably this ought to be a special case in PyErr_SetFromWindowsErrWithFilename(). And either we need to search the string for the special names or find an API that will clarify it (GetFileAttributes?).

I definitely do not want every error message suggesting that this edge case may be the cause, and I don't want to prevent people using these names or purpose for their actual uses.
msg347610 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-07-10 08:38
> if the operation succeeds, no error/message is displayed

Some errors pass silently and may be confusing later on. For example, CreateDirectoryW calls NtCreateFile with the disposition FILE_CREATE and the option FILE_DIRECTORY_FILE (i.e. the call *must* create a new directory), but some non-filesystem devices ignore this and let the request succeed. For example:

    >>> os.mkdir('C:/Temp/nul')
    >>> os.mkdir('C:/Temp/conin$')

Both calls 'succeed' but don't actually create a directory:

    >>> os.path.exists(r'\\?\C:\Temp\nul')
    False
    >>> os.path.exists(r'\\?\C:\Temp\conin$')
    False

> either we need to search the string for the special names or find 
> an API that will clarify it

GetFullPathNameW is a library function that shares the implementation that's used to normalize paths in a create or open context. 

If the path does not start with \\?\, then we resolve it via GetFullPathNameW. If it becomes a device path, or if the final component changes (e.g. a trailing dot or space is removed), then we can parenthetically include the resolved path. For example:

os.stat('spam.'):

    FileNotFoundError: [WinError 2] The system cannot find the file specified: 'spam.' [resolved path: 'C:\\Temp\\spam']

os.startfile('con'):

    OSError: [WinError 1200] The specified device name is invalid: 'con' [resolved path: '\\\\.\\con']

os.open('C:/Temp/lpt9', 0):

    FileNotFoundError: [Errno 2] No such file or directory: 'C:/Temp/lpt9' [resolved path: '\\\\.\\lpt9']

For os.open() and open(), we should switch to using PyErr_SetExcFromWindowsErrWithFilenameObject instead of PyErr_SetFromErrnoWithFilenameObject. _doserrno is valid in this context, so we should be using it anyway to retain the more specific winerror value. Then we only have to special case reserved names in the Windows-only function PyErr_SetExcFromWindowsErrWithFilenameObjects.
History
Date User Action Args
2019-07-10 08:38:13eryksunsetkeywords: + 3.6regression

messages: + msg347610
2019-07-09 10:11:08steve.dowersetmessages: + msg347537
2019-07-07 04:03:10eryksunsetmessages: + msg347465
2019-07-07 01:11:27steven.dapranosettype: enhancement
2019-07-07 00:00:34steven.dapranocreate