classification
Title: os.path.normcase documentation/behaviour unclear on Mac OS X
Type: behavior Stage: resolved
Components: macOS Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, ezio.melotti, ijmorlan, loewis, matthieu.labbe, ned.deily, python-dev, ronaldoussoren
Priority: normal Keywords: patch

Created on 2008-08-01 17:16 by ijmorlan, last changed 2014-06-18 20:12 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
normcase.patch ronaldoussoren, 2009-03-30 14:27 review
Messages (13)
msg70571 - (view) Author: Isaac Morland (ijmorlan) Date: 2008-08-01 17:16
$ python
Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from os.path import normcase
>>> normcase ('aB')
'aB'
>>>

From http://docs.python.org/lib/module-os.path.html:  "On Unix, this
returns the path unchanged; on case-insensitive filesystems, it converts
the path to lowercase."  Of course, Mac OS X is both Unix and
case-insensitive, which is a rather bizarre combination, but that's it
is.  Where is the item for "make all file systems case-sensitive, put
the case-insensitivity in the user interface"?
msg70584 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2008-08-01 19:01
"Of course, Mac OS X is both Unix and case-insensitive"

Not so. Case-{in|}sensitivity is an attribute of HFS+ file systems that is 
specifiable when a file system is created; it's true that the default is 
still case-insensitive. There are also other case-sensitive file systems 
supported on OS X.
msg70586 - (view) Author: Isaac Morland (ijmorlan) Date: 2008-08-01 19:19
Ok, good point.  Perhaps the documentation should be updated to clarify
that Mac OS is treated as Unix even though a default Mac OS X
installation will have a case-insensitive file system.  Wouldn't it be
possible for a Windows machine to have a case-sensitive file system
similarly?  Or is it more "built-in" that the file system on a Windows
box will be case-insensitive?
msg70599 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-08-01 21:56
On Windows, case-insensitivity is part of the API, not of the file
system. NTFS itself is case-sensitive, and there are case-sensitive
subsystems on top of it (e.g. the POSIX subsystem, Interix). Win32, when
calling the file system, asks for case-insensitive lookup (which NTFS
also supports efficiently).

I believe FAT is inherently case-insensitive.
msg84549 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-03-30 14:27
The following patch explicitly mentions that os.path.normcase treats OSX 
like any other Unix platform.

I'm in favor of applying this.
msg84631 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-03-30 20:06
I've committed a fix for this in r70746  and ported this to 3.1 and 2.6 as 
well.
msg108607 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2010-06-25 15:42
I'm reopening this because now the normcase implementation of posixpath has a todo that says "on Mac OS X, this should really return s.lower()." (see r66743).
There is some discussion about this in #9018.
msg108794 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-06-27 14:40
The comment is wrong, os.path.normcase should not return s.lower(), as the other issue notes the case sensitivity is filesystem dependent.  

Returning s.lower() will break code when dealing with paths that refer to an NFS share on a Linux server, or to a path on a local case sensitive file system.

IMHO The comment in posixpath should be changed to something like this:

# On MacOS X most but not all filesystems are case sensitive, which
# means that using normcase(p1) != normcase(p2) doesn't necessarily mean
# that the paths refer to different files.

The current comment gives the impression that a trivial should be applied, which isn't true.
msg108804 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2010-06-27 19:47
Changing the function is better than changing the comment.
Although changing just the comment is better than leaving it unchanged.
msg109470 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-07-07 11:57
Note that this does not just affect MacOSX, this even affects Linux (although the behaviour on the default filesystems is different).

On Linux the default is case-sensitive filesystems, but you can use network filesystems that have case-insensitive behavior, for example by using a CIFS share. 

This means that /mnt/myserver-share/foo/bar and /mnt/myserver-share/FOO/BAR may or may not refer to the same "object": there aren't the same if /mnt/myserver-share is a regular Linux filesystem but will be when /mnt/myserver-share is a mount from a CIFS server.

In short: os.path.normcase is hopelessly naive in its assumptions about system behavior.
msg220949 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-18 19:56
Is this or could this be addressed via the new pathlib module?
msg220950 - (view) Author: Roundup Robot (python-dev) Date: 2014-06-18 20:11
New changeset a854d23305de by Ned Deily in branch '3.4':
Issue #3485: remove misleading comment
http://hg.python.org/cpython/rev/a854d23305de

New changeset 3edda677119e by Ned Deily in branch 'default':
Issue #3485: merge from 3.4
http://hg.python.org/cpython/rev/3edda677119e
msg220951 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-06-18 20:12
The misleading "TODO" comment has been removed.
History
Date User Action Args
2014-06-18 20:12:35ned.deilysetstatus: open -> closed
resolution: accepted -> fixed
messages: + msg220951

stage: resolved
2014-06-18 20:11:48python-devsetnosy: + python-dev
messages: + msg220950
2014-06-18 19:56:15BreamoreBoysetnosy: + BreamoreBoy
messages: + msg220949
2010-07-08 15:00:50matthieu.labbesetnosy: + matthieu.labbe
2010-07-07 11:57:39ronaldoussorensetmessages: + msg109470
2010-06-27 19:47:24ezio.melottisetmessages: + msg108804
versions: + Python 3.2, - Python 2.5
2010-06-27 14:40:52ronaldoussorensetmessages: + msg108794
2010-06-25 15:42:58ezio.melottisetstatus: closed -> open
nosy: + ezio.melotti
messages: + msg108607

2009-03-30 20:06:36ronaldoussorensetstatus: open -> closed
resolution: accepted
messages: + msg84631
2009-03-30 14:27:34ronaldoussorensetfiles: + normcase.patch
versions: + Python 2.6, Python 3.1, Python 2.7
nosy: + ronaldoussoren

messages: + msg84549

keywords: + patch
2008-08-01 21:56:15loewissetnosy: + loewis
messages: + msg70599
2008-08-01 19:19:49ijmorlansetmessages: + msg70586
2008-08-01 19:01:52ned.deilysetnosy: + ned.deily
messages: + msg70584
2008-08-01 17:16:04ijmorlancreate