Author eryksun
Recipients Charles Machalow, eric.smith, eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Date 2020-03-12.11:04:47
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1584011088.45.0.746539536584.issue39920@roundup.psfhosted.org>
In-reply-to
Content
> This is an os.lstat() limitation, nothing to do with pathlib.

In 3.8+, stat() and lstat() work with a broader rang of device paths, at least for st_mode checks. For example:

    >>> s = os.stat('//./PhysicalDrive0')
    >>> stat.S_ISBLK(s.st_mode)
    True

The problem is pathlib. It handles device paths improperly, such as normalizing '//./PhysicalDrive0' to add a trailing backslash. For example:

    >>> p = Path('//./PhysicalDrive0')
    >>> os.fspath(p)
    '\\\\.\\PhysicalDrive0\\'

Issue 33898 covers the issue in more detail and has a pending PR. IMO, the PR needs work (e.g. correct handling of repeated slashes in UNC and device paths, IIRC). It doesn't have many people clamoring for a fix and stepping up to help. I'll try to get back to it soon -- when I'm up to it.

---

**Discussion of the observed error**

"\\.\PhysicalDrive<N>" is an object symlink to a native Device object [1] that represents a physical disk device. Typically the native path is something like "\Device\Harddisk<X>\DR<Y>". This is aliased as "\Device\Harddisk<X>\Partition0", i.e. the 0th partition is the entire disk. A fixed disk will be partitioned into one or more volume devices such as "\Device\HarddiskVolume<N>". Note that native volume device paths are transient, depending on PNP device removal and insertion. To work around this, the Mountpoint Manager maps the native path to a persistent "Volume{GUID}" name, and usually also a persistent DOS drive name such as "X:".

Disk and volume devices have a Volume Parameter Block (VPB) [2], which allows a filesystem device to mount them. The filesystem device manages the device namespace and acts as a proxy when accessing the device. In contrast, other device types have no VPB and thus directly manage their own namespace. (A device may implement full or partial support for filesystem functionality, without being mounted, such as "\\.\PIPE", i.e. "\Device\NamedPipe".) 

The VPB of a disk device is flagged VPB_RAW_MOUNT, which restricts it to the RAW filesystem. (A volume device that isn't formatted or doesn't have a recognizable filesystem will also use RAW.) The RAW filesystem device supports no namespace. Any remaining path, including a root path, fails with STATUS_INVALID_PARAMETER (0xC000_000D), i.e. Windows ERROR_INVALID_PARAMETER (87). 

In order to observe this error condition in action, the disk device has to first be mounted by RAW. This is possible with a Direct Access Storage Device (DASD) open, for which there is no remaining path and which requests more than just reading device metadata (attributes, security), such as requesting data access. 

The OP sees a different error because the disk isn't already mounted. This short circuits to the default result of STATUS_UNSUCCESSFUL (0xC000_0001), i.e. Windows ERROR_GEN_FAILURE (31). 

Here's an example to clarify the behavior.

In this first case, the VPB isn't mounted yet by RAW, and mounting fails since the device path includes a remaining path (i.e. a single trailing "/"):

    >>> try: os.stat('//./PhysicalDrive0/')
    ... except OSError as e: print(e)
    ...
    [WinError 31] A device attached to the system is not functioning


If one first mounts the disk with RAW by way of a DASD open, then subsequently one sees the expected invalid parameter error for a non-DASD open:

    >>> f = open('//./PhysicalDrive0', 'rb')

    >>> try: os.stat('//./PhysicalDrive0/')
    ... except OSError as e: print(e)
    ...
    [WinError 87] The parameter is incorrect: '//./PhysicalDrive0/'


[1]: https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ns-wdm-_device_object
[2]: https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ns-wdm-_vpb
History
Date User Action Args
2020-03-12 11:04:48eryksunsetrecipients: + eryksun, paul.moore, eric.smith, tim.golden, zach.ware, steve.dower, Charles Machalow
2020-03-12 11:04:48eryksunsetmessageid: <1584011088.45.0.746539536584.issue39920@roundup.psfhosted.org>
2020-03-12 11:04:48eryksunlinkissue39920 messages
2020-03-12 11:04:47eryksuncreate