classification
Title: tempfile.gettempdir() didn't return the path with correct case.
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: tim.golden Nosy List: brian.curtin, georg.brandl, jteh, mcjeff, ncoghlan, pitrou, python-dev, shimizukawa, tim.golden, 勇刚.罗
Priority: low Keywords: patch

Created on 2012-03-12 01:12 by 勇刚.罗, last changed 2013-10-28 10:32 by tim.golden. This issue is now closed.

Files
File name Uploaded Description Edit
tempfile_normpath.patch mcjeff, 2012-03-15 11:21 review
issue14255.3.diff tim.golden, 2013-10-25 19:10 review
Messages (20)
msg155424 - (view) Author: 勇刚 罗 (勇刚.罗) Date: 2012-03-12 01:12
>>> print tempfile.gettempdir()
c:\users\dreamkxd\appdata\local\temp
>>>
And the real path is
C:\Users\dreamkxd\AppData\Local\Temp.
msg155717 - (view) Author: Jeff McNeil (mcjeff) * Date: 2012-03-14 03:40
The actual implementation calls os.path.normcase from tempfile._get_default_tempdir. In this scenario here, that resolves to the ntpath module & triggers a lowercase conversion.

On the other hand, the posixpath module is simply an identity function.

Now, it *is* possible to force a Windows file system to run in a case-sensitive configuration. Since ntpath.py forces lower case, this could actually cause breakage in that situation, however rare.

In my opinion, changing the ntpath.normcase function to retain case probably yields more correct behavior.
msg155736 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-03-14 07:31
There's not much sense in having normcase() if it never does anything :)

But in this case, I don't really see why tempfile needs to use normcase().
msg155745 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-14 12:04
+1 for fixing.
msg155769 - (view) Author: Jeff McNeil (mcjeff) * Date: 2012-03-14 17:01
The normcase call isn't exactly a no-op in ntpath.py as it also swaps / for \\.  Removing the .lower() seems to simply make it a watered down version of normpath.

There are a couple of options I guess.  We could simply keep the altsep, or update _get_default_tempdir to use normpath?  

I'd be happy to do it once everyone agrees =)
msg155861 - (view) Author: 勇刚 罗 (勇刚.罗) Date: 2012-03-15 05:45
+1
 _get_default_tempdir to use normpath
msg155875 - (view) Author: Jeff McNeil (mcjeff) * Date: 2012-03-15 11:21
Here's a tiny patch that just changes normcase->normpath.  This fixes the casing issue at the 'gettempdir' level, though it doesn't address the 'normcase' function itself.

Note that *both* macpath.py and ntpath.py use <s>.lower, which obviously won't fly on case-sensitive mounts. If someone more experienced than I has a suggestion on how to handle that -- or whether its worthwhile to even touch -- I'll gladly implement.
msg155892 - (view) Author: 勇刚 罗 (勇刚.罗) Date: 2012-03-15 15:05
It's possible to getting actual file name under Windows.
http://stackoverflow.com/questions/74451/getting-actual-file-name-with-proper-casing-on-windows
msg155949 - (view) Author: Jeff McNeil (mcjeff) * Date: 2012-03-15 20:06
It doesn't seem to me that the right approach on either platform is to simply downcase.  Maybe just deprecate the call altogether as its not doing anything normpath isn't if the <s>.lower call is removed?
msg173307 - (view) Author: James Teh (jteh) Date: 2012-10-19 02:27
This issue is much nastier than it seems when dealing with character sets that contain multi-byte characters in Python 2.7 on Windows. For example, on a Japanese system:
1. The ANSI code page is cp932 and Python 2.7 will return the TEMP environment variable as a str with this encoding.
2. If a user has a username containing the character \u5c71, this will be encoded as \x8eR, so the TEMP environment variable will include this character.
3. When normcase is called, \x8eR is converted to \x8er.
4. Because \x8eR is a multi-byte character, \x8er is an unrelated character; case insensitivity for "R" doesn't apply.
5. The result is that the correct temp directory is skipped. In fact, on most systems, IOError is raised because none of the other candidate directories are valid either.
msg173426 - (view) Author: Jeff McNeil (mcjeff) * Date: 2012-10-21 02:48
Yeah clearly the wrong behavior on Winders.

I think that moving to 'normpath' instead of 'normcase' is likely the right thing to do. Patch is attached, so if someone with commit powers could review real quick I'll address whatever needs to be addressed.

IMHO, the 'normcase' calls are of questionable benefit since the posix call is simply an identity and as this bug details, it isn't necessarily the right behavior on Win.
msg180190 - (view) Author: Takayuki SHIMIZUKAWA (shimizukawa) Date: 2013-01-18 13:16
> I think that moving to 'normpath' instead of 'normcase'

Official manual says 'abspath' include 'normpath' functionality.
I think it is need JUST remove '_os.path.normcase' calling.

But, I do not understand the circumstances which use 'normcase'.
msg201125 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-10-24 13:25
normpath doesn't really buy anything here as abspath already has the same effect (plus more). Patch attach removes normcase|path leaving only abspath.
msg201126 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-10-24 13:25
If no-one objects, I'll commit in a day or two.
msg201147 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-24 15:05
Could you perhaps add a test?
msg201168 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-10-24 19:49
Added, including a slightly surprising change needed to test_zipimport_support (which arguably should have been there from the start for robustness).
msg201169 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-24 19:54
The test fails under Linux here:

======================================================================
FAIL: test_case_sensitive (test.test_tempfile.TestGetTempDir)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_tempfile.py", line 490, in test_case_sensitive
    self.assertEqual(tempfile.gettempdir(), case_sensitive_tempdir)
AssertionError: '/tmp' != 'C:\\Temp'
- /tmp
+ C:\Temp
msg201277 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-10-25 19:10
New patch, tested on Windows & Linux
msg201480 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-10-27 17:51
New changeset d5a9a1ba47ee by Tim Golden in branch 'default':
Issue14255 Don't flatten case of tempdir
http://hg.python.org/cpython/rev/d5a9a1ba47ee
msg201520 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-10-28 10:32
Fixed. Thanks for the report.
History
Date User Action Args
2013-10-28 10:32:31tim.goldensetstatus: open -> closed
resolution: fixed
messages: + msg201520

stage: patch review -> resolved
2013-10-27 17:51:01python-devsetnosy: + python-dev
messages: + msg201480
2013-10-25 19:10:37tim.goldensetfiles: + issue14255.3.diff

messages: + msg201277
2013-10-25 19:09:59tim.goldensetfiles: - issue14255.2.diff
2013-10-24 19:54:11pitrousetmessages: + msg201169
2013-10-24 19:49:35tim.goldensetfiles: + issue14255.2.diff
assignee: tim.golden
messages: + msg201168
2013-10-24 19:47:53tim.goldensetfiles: - issue14255.diff
2013-10-24 15:05:40pitrousetstage: needs patch -> patch review
messages: + msg201147
components: - Windows, IO
versions: + Python 3.4, - Python 3.2
2013-10-24 13:25:54tim.goldensetfiles: + issue14255.diff

messages: + msg201126
2013-10-24 13:25:22tim.goldensetmessages: + msg201125
2013-01-18 13:16:21shimizukawasetnosy: + shimizukawa
messages: + msg180190
2012-10-21 02:48:23mcjeffsetmessages: + msg173426
2012-10-19 02:27:57jtehsetnosy: + jteh
messages: + msg173307
2012-03-15 20:06:42mcjeffsetmessages: + msg155949
2012-03-15 15:05:52勇刚.罗setmessages: + msg155892
2012-03-15 11:21:53mcjeffsetfiles: + tempfile_normpath.patch
keywords: + patch
messages: + msg155875
2012-03-15 05:45:44勇刚.罗setmessages: + msg155861
2012-03-14 17:01:35mcjeffsetmessages: + msg155769
2012-03-14 12:04:36pitrousetpriority: normal -> low

nosy: + pitrou
messages: + msg155745

stage: test needed -> needs patch
2012-03-14 07:31:53georg.brandlsetmessages: + msg155736
2012-03-14 03:40:55mcjeffsetnosy: + mcjeff
messages: + msg155717
2012-03-13 02:36:54eric.araujosetnosy: + georg.brandl, ncoghlan, tim.golden, brian.curtin
stage: test needed
type: behavior

versions: - Python 2.6, Python 3.1, Python 3.4
2012-03-12 01:13:30勇刚.罗setcomponents: + Library (Lib), Windows, IO
versions: + Python 2.6, Python 3.1, Python 2.7, Python 3.2, Python 3.3, Python 3.4
2012-03-12 01:12:42勇刚.罗create