This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: os.path.normcase gets fooled on windows with mapped linux network drive
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: dawidjoubert, eryksun, ezio.melotti, mark, steve.dower, tim.golden, valhallasw, zach.ware
Priority: normal Keywords:

Created on 2008-10-24 22:03 by dawidjoubert, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (7)
msg75187 - (view) Author: dawidjoubert (dawidjoubert) Date: 2008-10-24 22:03
The documentation for os.path.normcase reads the case gets normalized
based on the file system
http://docs.python.org/dev/library/os.path.html

Current documentation:
Normalize the case of a pathname. On Unix, this returns the path
unchanged; on case-insensitive filesystems, it converts the path to
lowercase. On Windows, it also converts forward slashes to backward slashes.

Problem:
When mounting a network drive (via Samba) onto windows the mapped
network drive will claim to be an NTFS network (on Windows XP). Where
the network drive is actually a Samba share this can cause problems with
case sensitivity versus case insensitivity.

More:
The Windows based file systems such as FAT, FAT32 and NTFS are as a
matter of fact case preservant but case insensitive. That is a file
saved with the name 'MyName' will retain its name as 'MyName' and a ls
(dir) command will returnr the file as 'MyName' however access to
'Myname' or 'myname' or 'MYNAME' will all still access the same file.

Solution:
I suggest we drop converting the case to lower case where the file
systems FAT, FAT32 and NTFS are involved
msg161296 - (view) Author: Merlijn van Deen (valhallasw) * Date: 2012-05-21 20:37
To confirm this behaviour is still current:

Python 3.3.0a1+ (default:958a98bf924e+, May 21 2012, 22:18:16)
[GCC 4.5.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ntpath
>>> ntpath.normcase(r"c:\mixedCASE")
'c:\\mixedcase'

In general, I'm not so sure of the use case of the function /at all/ - possibly something like
  
if normcase(path1) == normcase(path2):
    ...

however, this can give the false impression that two files are the same - I think, given a Samba share with two files 'A' and 'a', it's possible to access both files from windows.
msg161354 - (view) Author: Merlijn van Deen (valhallasw) * Date: 2012-05-22 14:20
OK, I did some quick tests. Given a samba share with the following files

A (contents: 'test: A')
a (contents: 'test: a')
B (contents: 'test: B')

1) opening \\share\files\A or \\share\files\a opens the same file - in my
case 'test: A'
2) opening \\share\files\B or \\share\files\b both work.

As such, the behaviour is at least not incorrect. In addition, it means
that the "if normcase(path1) == normcase(path2):" option to check if two
files are the same is actually correct (if normcase(path1) ==
normcase(path2) then open(path1).read() == open(path2).read() - at least on
the share I tested). Interestingly, os.path.samefile suggests this would
not be the case:

True
>>> os.path.samefile("u:\\a", "u:\\A")
False

On 22 May 2012 03:59, Ezio Melotti <report@bugs.python.org> wrote:

>
> Changes by Ezio Melotti <ezio.melotti@gmail.com>:
>
>
> ----------
> nosy: +brian.curtin, tim.golden
> stage:  -> needs patch
> versions:  -Python 3.4
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue4198>
> _______________________________________
>
msg224250 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-29 20:58
How does the new pathlib module handle this?  Also note from the docs for os.samefile() "Changed in version 3.4: Windows now uses the same implementation as all other platforms.".
msg345846 - (view) Author: Mark Summerfield (mark) * Date: 2019-06-17 11:40
When running a VirtualBox Windows 7 guest on Linux I have found that os.path.samefile() returns False when the filenames are the same, e.g.,

f = r'V:\pdfs\boson1.pdf'
same = os.path.samefile(f, f)
print(same) # expected True; got False

I mention it here because it may be part of the same problem.
I'm using Python 3.7.2.
msg346010 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-06-18 21:14
> When running a VirtualBox Windows 7 guest on Linux I have found 
> that os.path.samefile() returns False when the filenames are the 
> same, e.g.,
>
> f = r'V:\pdfs\boson1.pdf'
> same = os.path.samefile(f, f)
> print(same) # expected True; got False

genericpath.samefile is unrelated to ntpath.normcase. It's based on comparing the st_dev and st_ino values from the os.stat result of each file. There are known problems with relying solely on the Windows implementation of os.stat for this. Please create a new issue to add your case to the list. Include details about the local volume or UNC share that's mounted as drive V:.
msg389217 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-03-21 05:20
Filesystem paths in Windows are normally opened case insensitive, but a filesystem can choose to ignore this. NTFS does so for directories that flagged as case sensitive [1]. ntpath.normcase() doesn't incorporate this information. It would have to query the case-sensitive information for each component in the path. Instead it just assumes that all path components are case insensitive. That said, the way it does this does not correspond to how Windows filesystems implement case-insensitive comparisons. See bpo-42658. If supporting case-sensitive directories is needed, it should be addressed in that issue.

> same = os.path.samefile(f, f)
> print(same) # expected True; got False

I just followed up on investigating this. Apparently the VirtualBox "VBoxSharedFolderFS" filesystem generates a different file reference number randomly on each access. This violates the requirements for FileInternalInformation as specified in [MS-FSCC][2]. If file reference numbers aren't supported, the filesystem is supposed to return 0. That said, Python's samefile() doesn't currently follow the spec either. It blindly uses the st_dev (volume serial number) and st_ino (file reference number) values even when they're 0. See bpo-33935.

---

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_case_sensitive_information
[2] https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/7d796611-2fa5-41ac-8178-b6fea3a017b3
History
Date User Action Args
2022-04-11 14:56:40adminsetgithub: 48448
2021-03-21 05:20:32eryksunsetstatus: open -> closed
resolution: not a bug
messages: + msg389217

stage: needs patch -> resolved
2019-06-18 21:14:49eryksunsetnosy: + eryksun
messages: + msg346010
2019-06-17 11:40:46marksetnosy: + mark

messages: + msg345846
versions: + Python 3.7, - Python 3.3
2019-04-26 20:10:13BreamoreBoysetnosy: - BreamoreBoy
2014-07-29 21:23:17brian.curtinsetnosy: - brian.curtin
2014-07-29 20:58:43BreamoreBoysetnosy: + BreamoreBoy, zach.ware, steve.dower
messages: + msg224250
2012-05-22 14:20:09valhallaswsetmessages: + msg161354
2012-05-22 01:59:28ezio.melottisetnosy: + tim.golden, brian.curtin
stage: needs patch

versions: - Python 3.4
2012-05-21 20:37:43valhallaswsetnosy: + valhallasw

messages: + msg161296
versions: + Python 3.3, Python 3.4, - Python 2.7
2010-06-25 12:18:51ezio.melottisetnosy: + ezio.melotti
2008-10-24 22:03:53dawidjoubertcreate