This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: os.stat() and os.lstat() fail with Windows device paths
Type: behavior Stage: resolved
Components: Library (Lib), Windows Versions: Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: pathlib issues with Windows device paths
View: 33898
Assigned To: Nosy List: Charles Machalow, eric.smith, eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2020-03-10 01:04 by Charles Machalow, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (6)
msg363796 - (view) Author: Charles Machalow (Charles Machalow) Date: 2020-03-10 01:04
I ran the following as admin in the Python interpreter (on Windows):


>>> d = pathlib.Path(r'\\.\PHYSICALDRIVE0')
>>> print(d)
\\.\PHYSICALDRIVE0\
>>> d.exists()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python37\lib\pathlib.py", line 1318, in exists
    self.stat()
  File "C:\Python37\lib\pathlib.py", line 1140, in stat
    return self._accessor.stat(self)
PermissionError: [WinError 31] A device attached to the system is not functioning: '\\\\.\\PHYSICALDRIVE0\\'
>>> d.is_char_device()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python37\lib\pathlib.py", line 1403, in is_char_device
    return S_ISCHR(self.stat().st_mode)
  File "C:\Python37\lib\pathlib.py", line 1140, in stat
    return self._accessor.stat(self)
PermissionError: [WinError 31] A device attached to the system is not functioning: '\\\\.\\PHYSICALDRIVE0\\'
>>> d.is_block_device()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python37\lib\pathlib.py", line 1390, in is_block_device
    return S_ISBLK(self.stat().st_mode)
  File "C:\Python37\lib\pathlib.py", line 1140, in stat
    return self._accessor.stat(self)
PermissionError: [WinError 31] A device attached to the system is not functioning: '\\\\.\\PHYSICALDRIVE0\\'

I think that exists(), is_char_device(), and is_block_device() should be able to work on Windows in some form or fashion. At least without a traceback.
msg363819 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-03-10 10:46
I know it's of limited use, but for me, this code works (returns True) on Cygwin versions 3.8.0b4 and 3.7.4.

On a native Windows build 3.8.0a0, I get the exception the OP shows.

I'll try and install comparable versions when I'm at a location with better connectivity.
msg363822 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-10 13:26
This is an os.lstat() limitation, nothing to do with pathlib.

I'll defer to Eryk for suggestions here, since he probably knows the right answer already :) (if not, I'll go do the research to figure out what we're currently missing).
msg364010 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-03-12 11:04
> This is an os.lstat() limitation, nothing to do with pathlib.

In 3.8+, stat() and lstat() work with a broader rang of device paths, at least for st_mode checks. For example:

    >>> s = os.stat('//./PhysicalDrive0')
    >>> stat.S_ISBLK(s.st_mode)
    True

The problem is pathlib. It handles device paths improperly, such as normalizing '//./PhysicalDrive0' to add a trailing backslash. For example:

    >>> p = Path('//./PhysicalDrive0')
    >>> os.fspath(p)
    '\\\\.\\PhysicalDrive0\\'

Issue 33898 covers the issue in more detail and has a pending PR. IMO, the PR needs work (e.g. correct handling of repeated slashes in UNC and device paths, IIRC). It doesn't have many people clamoring for a fix and stepping up to help. I'll try to get back to it soon -- when I'm up to it.

---

**Discussion of the observed error**

"\\.\PhysicalDrive<N>" is an object symlink to a native Device object [1] that represents a physical disk device. Typically the native path is something like "\Device\Harddisk<X>\DR<Y>". This is aliased as "\Device\Harddisk<X>\Partition0", i.e. the 0th partition is the entire disk. A fixed disk will be partitioned into one or more volume devices such as "\Device\HarddiskVolume<N>". Note that native volume device paths are transient, depending on PNP device removal and insertion. To work around this, the Mountpoint Manager maps the native path to a persistent "Volume{GUID}" name, and usually also a persistent DOS drive name such as "X:".

Disk and volume devices have a Volume Parameter Block (VPB) [2], which allows a filesystem device to mount them. The filesystem device manages the device namespace and acts as a proxy when accessing the device. In contrast, other device types have no VPB and thus directly manage their own namespace. (A device may implement full or partial support for filesystem functionality, without being mounted, such as "\\.\PIPE", i.e. "\Device\NamedPipe".) 

The VPB of a disk device is flagged VPB_RAW_MOUNT, which restricts it to the RAW filesystem. (A volume device that isn't formatted or doesn't have a recognizable filesystem will also use RAW.) The RAW filesystem device supports no namespace. Any remaining path, including a root path, fails with STATUS_INVALID_PARAMETER (0xC000_000D), i.e. Windows ERROR_INVALID_PARAMETER (87). 

In order to observe this error condition in action, the disk device has to first be mounted by RAW. This is possible with a Direct Access Storage Device (DASD) open, for which there is no remaining path and which requests more than just reading device metadata (attributes, security), such as requesting data access. 

The OP sees a different error because the disk isn't already mounted. This short circuits to the default result of STATUS_UNSUCCESSFUL (0xC000_0001), i.e. Windows ERROR_GEN_FAILURE (31). 

Here's an example to clarify the behavior.

In this first case, the VPB isn't mounted yet by RAW, and mounting fails since the device path includes a remaining path (i.e. a single trailing "/"):

    >>> try: os.stat('//./PhysicalDrive0/')
    ... except OSError as e: print(e)
    ...
    [WinError 31] A device attached to the system is not functioning


If one first mounts the disk with RAW by way of a DASD open, then subsequently one sees the expected invalid parameter error for a non-DASD open:

    >>> f = open('//./PhysicalDrive0', 'rb')

    >>> try: os.stat('//./PhysicalDrive0/')
    ... except OSError as e: print(e)
    ...
    [WinError 87] The parameter is incorrect: '//./PhysicalDrive0/'


[1]: https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ns-wdm-_device_object
[2]: https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ns-wdm-_vpb
msg364070 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-03-13 03:13
The particular error reported here is due to pathlib, which is a duplicate of issue 33898. That said, since the OP is using 3.7, I've left this issue open in case someone wants to backport the win32_xstat_impl changes that enable stat() (and in particular setting st_mode) for Windows 'character' (e.g. //./NUL and //./CONIN$) and 'block' devices (e.g. //./C: and //./PhysicalDrive0). I don't recall why Steve Dower decided against backporting this update.
msg389267 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-03-22 01:00
3.7.8, released on 2020-06-27, was the last bugfix release of Python 3.7, so I'm closing this issue as a duplicate of bpo-33898, which is about supporting device paths correctly in pathlib.
History
Date User Action Args
2022-04-11 14:59:27adminsetgithub: 84101
2021-03-22 01:00:39eryksunsetstatus: open -> closed
superseder: pathlib issues with Windows device paths
messages: + msg389267

resolution: duplicate
stage: resolved
2020-03-13 03:13:10eryksunsettitle: os.lstat() does not work with Window Dos devices -> os.stat() and os.lstat() fail with Windows device paths
messages: + msg364070
components: + Library (Lib)
versions: - Python 3.8, Python 3.9
2020-03-12 11:04:48eryksunsetmessages: + msg364010
2020-03-10 13:26:37steve.dowersetversions: + Python 3.7, Python 3.9
2020-03-10 13:26:31steve.dowersetmessages: + msg363822
title: Pathlib path methods do not work with Window Dos devices -> os.lstat() does not work with Window Dos devices
2020-03-10 10:54:10xtreaksetnosy: + eryksun
2020-03-10 10:46:06eric.smithsetnosy: + paul.moore, tim.golden, eric.smith, zach.ware, steve.dower
messages: + msg363819
components: + Windows
2020-03-10 01:04:55Charles Machalowcreate