This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eryksun
Recipients eryksun, izbyshev, miss-islington, paul.moore, steve.dower, tim.golden, zach.ware
Date 2018-03-11.10:20:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1520763660.54.0.467229070634.issue33016@psf.upfronthosting.co.za>
In-reply-to
Content
> pathlib currently expects DOS paths only: it will strip '\\?\' 
> prefix in resolve()

pathlib's unqualified conversion from the extended form to classic DOS form is a bug. The resolved path may be invalid as a classic DOS path. It may be too long, a DOS device name (e.g. "nul.txt"), or end with trailing spaces or dots (e.g. "spam."). 

Here's another pathlib bug with device paths:

    >>> print(pathlib.Path(r'\\.\C:'))
    \\.\C:\
    >>> print(pathlib.Path('//./CON'))
    \\.\CON\

In the first case, the input path is the "C:" volume device, but pathlib changes it to the file-system root directory. The second case is invalid in general, though some devices will ignore a remaining path.

As background, note that the Windows runtime library classifies paths into 6 types:

    >>> ntdll = ctypes.WinDLL('ntdll')
    >>> GetDosPathNameType = ntdll.RtlDetermineDosPathNameType_U

    UNC Absolute
    >>> GetDosPathNameType(r'\\eggs\spam')
    1

    Drive Absolute
    >>> GetDosPathNameType(r'C:\spam')
    2

    Drive Relative
    >>> GetDosPathNameType('C:spam')
    3

    Rooted
    >>> GetDosPathNameType(r'\spam')
    4

    Relative
    >>> GetDosPathNameType('spam')
    5

    Local Device
    >>> GetDosPathNameType(r'\\.\C:\spam')
    6
    >>> GetDosPathNameType(r'\\?\C:\spam')
    6

A local-device path is always absolute, so it's the only way to reference a volume device by a DOS drive letter. Without the prefix, "C:" is a drive-relative path.

If the prefix is exactly "\\\\?\\" (no forward slashes), then a local-device path is an extended path. This path type never gets normalized in any way, except to replace the WinAPI prefix with NTAPI "\\??\\". For all other local-device paths, the runtime library resolves "." and ".." components, translates forward slash to backslash, and strips trailing spaces and dots from the final component. Unlike DOS paths, local-device paths do not reserve DOS device names (e.g. "NUL" or "NUL:").

pathlib should never add a trailing slash to a local-device path. Also, the is_reserved() method needs to distinguish the DOS, device ("\\\\.\\"), and extended device ("\\\\?\\") namespaces.

> it would indeed be nice if pathlib handled [device] paths in its resolve()

I suggested handling volume GUID and device paths in _getfinalpathname, so it's not special-cased just in pathlib (e.g. if we were to implement ntpath.realpath). OTOH, pathlib's resolve() method should handle high-level mitigation, such as working around bad links and permission errors in non-strict mode.

> I erroneously stated that the length of the path could increase 
> between GetFinalPathNameByHandle calls because an intermediate
> directory could be renamed

The rules for the rename operation are discussed in the documentation for NtSetInformationFile, FileRenameInformation [1], and explained in detail in File Systems Algorithms, 2.1.5.14.11 FileRenameInformation [2].

[1]: https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/ns-ntifs-_file_rename_information
[2]: https://msdn.microsoft.com/en-us/library/ff469527
History
Date User Action Args
2018-03-11 10:21:01eryksunsetrecipients: + eryksun, paul.moore, tim.golden, zach.ware, steve.dower, izbyshev, miss-islington
2018-03-11 10:21:00eryksunsetmessageid: <1520763660.54.0.467229070634.issue33016@psf.upfronthosting.co.za>
2018-03-11 10:21:00eryksunlinkissue33016 messages
2018-03-11 10:20:57eryksuncreate