This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [Windows] require an existing directory and suport junctions in ntpath.ismount()
Type: behavior Stage:
Components: Library (Lib), Windows Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: 37609 Superseder:
Assigned To: Nosy List: Jonathan Hsu, eryksun, jainankur, lazka, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2019-12-01 10:40 by jainankur, last changed 2022-04-11 14:59 by admin.

Messages (7)
msg357674 - (view) Author: Ankur (jainankur) Date: 2019-12-01 10:40
Tested with following lines of code on windows 10:

import os.path
print(os.path.ismount("F:"))

The above statement returns true in python 3.7.4 and false in 2.7.14

Note that F: drive does not have any mount points.
Somehow, python 3.7.4 returns true for all drive letters except C:
msg357675 - (view) Author: Christoph Reiter (lazka) * Date: 2019-12-01 10:50
related: issue28859
msg357722 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-12-02 17:51
ntpath.ismount() is using a name-only check here, it never tries to resolve the path if splitdrive() returns (anything, one_of("", "/", "\\"))

Now it's not clear whether handling "" is intended there, but that's certainly the difference (Python 2 required exactly one character in that part). And it appears to be consistent back to 3.4, so I'm not sure we can or should fix anything. The only thing 'F:' can ever be is a mount point.

However, we should probably fix the fact that C: returns False. This is because we call abspath(path), and "x:" is a relative path. When the drive does not match the current working drive, you get "x:\\". But when it _does_ match, you get the current working directory, which (unless it's the root directory) is no longer a mount point.

Perhaps the best fix for that is to just say that "x:" is a relative path and hence never a mount point? Which would be the same as switching back to the Python 2 behaviour, but would be a much better reason.
msg357892 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-12-05 23:09
> The only thing 'F:' can ever be is a mount point.

"F:" is a relative path that has to be resolved via abspath() (i.e. GetFullPathNameW in Windows) before we can determine whether it's a mount point. ntpath.ismount handles this correctly in 3.4+. 

The working directory on drive F: is the process working directory if it's the same drive, else it's the value of the "=F:" environment variable if defined, else it defaults to the root directory. The Windows API uses "=X:" environment variables, but it does not create them. They're optionally created by applications (e.g. CMD sets them, as does Python's os.chdir). Since they're just environment variables (albeit hidden because they begin with "="), per-drive working directories are inherited by child processes by default.

The main problem that I see here is that, in Windows (not POSIX), ntpath.ismount fails to verify that a potential mount point is an existing directory via os.stat and stat.S_ISDIR. This includes junctions that potentially target non-existing volumes (e.g. a junction to a removable volume that's unavailable or a volume with removable media that's not ready). This would be a breaking change, but I think it's important enough. Returning true for non-existing 'mount points' is nonsensical, since no filesystem actually mounts the path and the path itself may not even exist (e.g. an undefined drive or device). It needs to be and existing path in a device namespace (a device namespace is not necessarily a filesystem; all devices in NT have their own namespace, with a root path, whether mounted or not), and it has to at least be a subdirectory of a filesystem, if not the root directory. 

A common example of a mount point that's not at the root path of a device is a UNC share. For example, "\\;LanmanRedirector\;X:<logon ID>\server\share" resolves to opening "\Device\Mup\;LanmanRedirector\;X:<logon ID>\server\share", which is the form that's used for mapping an SMB share as a drive, except that a mapped drive can also target a subdirectory of a share. The opened path includes an explicit reference to the redirector device (to bypass prefix resolution), the drive and logon session for the cached connection (to bypass connecting), and an arbitrary number of prefix components (typically just "server\share"). 

Another related concern is the test for a root directory, `rest in seps`, which makes ntpath.ismount("F:") true in POSIX for the wrong reason. Since `seps` is a string, this check should actually be `rest and rest in seps' in order to avoid the vacuous truth when `rest` is an empty string. This problem is particular to POSIX because ntpath._abspath_fallback mishandles drive-relative paths. For example, ntpath.abspath("F:") returns "F:" in POSIX. It should special case a drive-relative path to insert a backslash (e.g. "F:" -> "F:\\", and "F:spam\\eggs" -> "F:\\spam\\eggs").

---

On the subject of junctions, a UNC path should not short-circuit out of checking for a junction, as it currently does. UNC paths can contain mount points, which get evaluated on the server.

In 3.8+, ntpath.ismount could also (and IMO should) support junctions that are bind mount points (i.e. mounting a subdirectory of a filesystem to a non-root path of a device) -- as posixpath.ismount does in POSIX. We already support bind mounts as subst and mapped drives and UNC shares, so it's just making ismount consistent, and for the better. This would require an os.lstat call to check st_file_attributes for stat.FILE_ATTRIBUTE_REPARSE_POINT and whether st_reparse_tag is stat.IO_REPARSE_TAG_MOUNT_POINT. It would still need an os.stat call and stat.S_ISDIR to verify that the target is an existing directory.
msg357893 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-12-05 23:33
> The main problem that I see here is that, in Windows (not POSIX), ntpath.ismount fails to verify that a potential mount point is an existing directory via os.stat and stat.S_ISDIR. ... This would be a breaking change, but I think it's important enough.

So essentially, you say the check should always be "ntpath.isdir(d) and ntpath.ismount(d)" (plus the non-breaking improvements to ismount)? And we should just do the isdir() inside ismount() because otherwise the result is nonsense.

I'm inclined to agree, and I'm not concerned about breaking a nonsense result. 

On the other hand, this will make ntpath.ismount meaningless on POSIX, as it will always require real access to the file system. Is it worth having a "pure" implementation for best effort in this case? Based on the splitdrive() patterns we support (plus the fixes to support them properly)?
msg357896 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-12-06 01:19
> So essentially, you say the check should always be "ntpath.isdir(d) 

I forgot that ntpath.isdir is now genericpath.isdir, so that will work. The old nt._isdir wouldn't work because it was equivalent to an lstat. Apparently a vestigial import of nt._isdir remains in ntpath.py, even though nt._isdir was removed:

Lib/ntpath.py#l677">https://hg.python.org/cpython/file/v3.Lib/ntpath.py#l677

> On the other hand, this will make ntpath.ismount meaningless on POSIX, 
> as it will always require real access to the file system. Is it worth 
> having a "pure" implementation for best effort in this case? 

I'm not suggesting an existence check in POSIX. Whoever uses ntpath in POSIX must be aware that ntpath.ismount is useless for detecting Unix mount points. It's only useful for identifying potential mount points in a pure Windows path. I see no reason to break this use case.

For POSIX, I'd like ntpath._abspath_fallback to handle drive-relative paths, so we can fix the check in ismount to use `rest and rest in seps`. It seems wrong for _abspath_fallback("F:") to return the relative path "F:". No drives exist in POSIX, so it's harmless to assume that the working directory on each drive is the root directory, which is the default value in Windows anyway.

> Based on the splitdrive() 

Note that splitdrive currently doesn't support "UNC" device paths (issue 37609). For example, "//?/UNC/server/share" splits as ("//?/UNC", "/server/share"). "GLOBAL" device paths also aren't supported, which I'd prefer, but it's less important because they're only useful rarely, and I don't know of an existing path library that handles them. splitdrive also doesn't support repeated separators between the server and share components, such as "//server///share", which GetFullPathNameW handles nowadays. Maybe it didn't circa NT 5.0. The only place that's strict is the UNC root, which must be exactly two separators. I uploaded an implementation to issue 37609 that supports "UNC" and "GLOBAL" device paths, with support for repeated separators. Here's a complex example split: ("//?///Global///UNC///server///share", "///path").

Anyway, ntpath.ismount needs to distinguish between UNC and "UNC" device paths vs all other device paths, because a root path is only optional in the UNC case. For example, "//?/C:" or "//?/BootPartition" is a volume device, and "//?C:/" or "//?/BootPartition/" is a filesystem mount point. In Windows we can detect that "//?/C:" or "//?/BootPartition" isn't a directory, but in POSIX we need to get it right as a pure path.
msg389628 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-03-28 02:09
Here's a concrete implementation of the above discussion. 

    _is_windows = (sys.platform == 'win32')

    def ismount(path):
        """Test whether a path is a mount point"""
        path = os.fspath(path)
        if isinstance(path, bytes):
            sep = b'\\'
            extended_devices = b'\\\\?\\'
            normal_devices = b'\\\\.\\'
            global_link = b'GLOBAL'
            unc_device = b'UNC'
        else:
            sep = '\\'
            extended_devices = '\\\\?\\'
            normal_devices = '\\\\.\\'
            global_link = 'GLOBAL'
            unc_device = 'UNC'
        # In Windows, require an existing, accessible directory.
        if _is_windows:
            if not isdir(path):
                return False
            try:
                s = os.lstat(path)
                if s.st_reparse_tag == stat.IO_REPARSE_TAG_MOUNT_POINT:
                    # isdir() verified the target directory.
                    return True
            except (OSError, ValueError):
                return False
        path = abspath(path)
        drive, drive_path = splitdrive(path)
        if not drive:
            return False
        # A drive root is a mount point.
        if drive_path == sep:
            return True
        # A root path is not required for a UNC drive.
        if not drive_path and drive[0] in sep:
            if not drive.startswith((extended_devices, normal_devices)):
                return True
            # Check for \\?\[Global]\UNC. Ignore repeated "Global" links.
            comps = [c.upper() for c in drive[4:].split(sep)]
            for c in comps:
                if c != global_link:
                    break
            if c == unc_device:
                return True
        return False

Removing the dependency on GetVolumePathNameW() also eliminates buggy behavior with substitute drives. For example, if substitute drive "W:" maps to r"C:\Windows", GetVolumePathNameW() mistakenly claims r"W:\System32" is a volume mount point.

In principle, this implementation also supports "\\?\[Global]\UNC\server\share" mount points on the "UNC" device, but it depends on fixing bpo-37609. The suggested rewrite for the latter issue also includes support for repeated slashes in a UNC drive, e.g. r"\\server\\\share" and r"\\?\\\Global\\\UNC\\\server\\\share" are valid.

For POSIX, ntpath._abspath_fallback() has to be fixed to correctly resolve drive-relative paths. For example:

    def _abspath_fallback(path):
        """Return the absolute version of a path."""
        path = os.fspath(path)
        if isinstance(path, bytes):
            sep = b'\\'
            colon = b':'
            cwd = os.getcwdb()
        else:
            sep = '\\'
            colon = ':'
            cwd = os.getcwd()
        if not isabs(path):
            # For a drive-relative path, default to the root directory
            # on the drive if the process working directory is on a
            # different drive.
            if path[1:2] == colon and path[:2].upper() != cwd[:2].upper():
                path = path[:2] + sep + path[2:]
            else:
                path = join(cwd, path)
        return normpath(path)

Since _abspath_fallback() is no longer needed in any version of Windows, maybe it should simply assume that the working drive is "C:" and the working directory on all drives is the root directory.
History
Date User Action Args
2022-04-11 14:59:23adminsetgithub: 83129
2021-03-28 02:09:36eryksunsetversions: + Python 3.10, - Python 3.7
title: os.path.ismount() returns False for current working drive -> [Windows] require an existing directory and suport junctions in ntpath.ismount()
messages: + msg389628

dependencies: + support "UNC" device paths in ntpath.splitdrive
components: + Library (Lib)
2020-03-23 22:11:02Jonathan Hsusetnosy: + Jonathan Hsu
2019-12-06 01:19:44eryksunsetmessages: + msg357896
2019-12-05 23:33:54steve.dowersetmessages: + msg357893
2019-12-05 23:09:02eryksunsetmessages: + msg357892
2019-12-02 17:51:36steve.dowersetnosy: + eryksun
title: os.path.ismount() returns true in python 3.7.4 and false in 2.7.14 -> os.path.ismount() returns False for current working drive
messages: + msg357722

versions: + Python 3.8, Python 3.9
type: behavior
2019-12-01 16:55:20ned.deilysetnosy: + paul.moore, tim.golden, zach.ware, steve.dower
components: + Windows, - 2to3 (2.x to 3.x conversion tool)
2019-12-01 10:50:39lazkasetnosy: + lazka
messages: + msg357675
2019-12-01 10:42:23jainankursettitle: os.module.ismount() returns true in python 3.7.4 and false in 2.7.14 -> os.path.ismount() returns true in python 3.7.4 and false in 2.7.14
2019-12-01 10:40:54jainankurcreate