classification
Title: PYTHONCASEOK is ignored on Windows
Type: behavior Stage: resolved
Components: Library (Lib), Windows Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: Brian.Herman, brett.cannon, eryksun, martin.panter, paul.moore, python-dev, rhettinger, serhiy.storchaka, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: easy

Created on 2016-05-22 06:45 by eryksun, last changed 2016-07-16 18:20 by brett.cannon. This issue is now closed.

Messages (13)
msg266057 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-05-22 06:45
importlib ignores the PYTHONCASEOK environment variable on Windows. _relax_case checks for b'PYTHONCASEOK' in os.environ, which is never true because os.environ is Unicode. On Windows, _make_relax_case should return a function that instead checks for 'PYTHONCASEOK'.

The following test demonstrates that case-insensitive imports otherwise appear to work correctly in 3.5:

    >>> import sys, importlib, glob
    >>> glob.glob('*.py')
    ['Test.py']

    >>> importlib._bootstrap_external._relax_case()
    False
    >>> import test
    >>> test.__file__
    'C:\\Program Files\\Python35\\lib\\test\\__init__.py'

patched:

    >>> src = "_relax_case = lambda: 'PYTHONCASEOK' in _os.environ"
    >>> exec(src, vars(importlib._bootstrap_external))
    >>> importlib._bootstrap_external._relax_case()
    True
    >>> del sys.modules['test']
    >>> import test
    this is a test
    >>> test.__file__
    'C:\\Temp\\test.py'

It would be better if __file__ were the actual filename "Test.py" instead of "test.py".
msg266302 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-25 04:18
Did you forgot to upload a patch?
msg270227 - (view) Author: Brian Herman (Brian.Herman) Date: 2016-07-12 05:16
In python 3.6 from hg it has been fixed.

>>> glob.glob('*.py')
['setup.py', 'test.py']
>>> import test
>>> test.__file__
'C:\\Users\\brian\\Desktop\\cpython\\test.py'
>>>
msg270499 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-07-15 18:55
New changeset bedcb9ec3f26 by Brett Cannon in branch '3.5':
Issue #27083: Respect the PYTHONCASEOK environment variable under
https://hg.python.org/cpython/rev/bedcb9ec3f26

New changeset 777da58794ca by Brett Cannon in branch 'default':
Merge for #27083
https://hg.python.org/cpython/rev/777da58794ca
msg270500 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-07-15 18:56
I fixed this by simply checking for both b'PYTHONCASEOK' and 'PYTHONCASEOK'. Thanks for the bug report, Eryk!
msg270506 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-15 19:47
Comparing bytes with str can emit a warning or raise an exception. Checking for b'PYTHONCASEOK' in os.environ looks as a bug to me.
msg270507 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-07-15 19:49
There's no direct comparison to trigger a warning; it's just seeing if a key in a dict exists.

And b'PYTHONCASEOK' is not a bug as that's required for OS X (remember this is technically posix.environ, not os.environ).
msg270508 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-15 19:55
Seeing if a key in a dict exists involves the key comparison if there is a key on the same position in the hash table.
msg270509 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-07-15 20:35
I think the best we can do is be platform-specific in what is checked for in the environ dict (e.g. b'PYTHONCASEOK' on darwin, 'PYTHONCASEOK' on win, and I don't know about cygwin), but otherwise one would have to iterate through the keys of the dict and do a type check before doing the comparison to fully avoid any potential warning being triggered.

Does the platform-specific decision of what to look for work for you, Serhiy?
msg270537 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-16 04:37
Yes, I think it will work.
msg270544 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-07-16 07:27
This appears to have broken the tests on Windows:

http://buildbot.python.org/all/builders/x86%20Windows7%203.5/builds/1030/steps/test/logs/stdio
======================================================================
FAIL: test_insensitive (test.test_importlib.source.test_case_sensitivity.Frozen_CaseSensitivityTestPEP302)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.5.bolen-windows7\build\lib\test\test_importlib\source\test_case_sensitivity.py", line 65, in test_insensitive
    self.assertIsNotNone(insensitive)
AssertionError: unexpectedly None

(Plus three more failures)
msg270576 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-07-16 17:46
New changeset 6b46c1510bfa by Brett Cannon in branch '3.5':
Fix regressions introduced by fixes for issue #27083.
https://hg.python.org/cpython/rev/6b46c1510bfa

New changeset f4c91b883772 by Brett Cannon in branch 'default':
Merge for #27083
https://hg.python.org/cpython/rev/f4c91b883772
msg270578 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-07-16 18:20
Buildbots are green again and I addressed the key type issue. Thanks for letting me know about the failure, Martin.
History
Date User Action Args
2016-07-16 18:20:12brett.cannonsetstatus: open -> closed

messages: + msg270578
stage: needs patch -> resolved
2016-07-16 17:46:19python-devsetmessages: + msg270576
2016-07-16 07:27:46martin.pantersetstatus: closed -> open
nosy: + martin.panter
messages: + msg270544

2016-07-16 04:37:30serhiy.storchakasetmessages: + msg270537
2016-07-15 20:35:14brett.cannonsetmessages: + msg270509
2016-07-15 19:55:54serhiy.storchakasetnosy: + rhettinger
messages: + msg270508
2016-07-15 19:49:47brett.cannonsetstatus: open -> closed

messages: + msg270507
2016-07-15 19:47:24serhiy.storchakasetstatus: closed -> open

messages: + msg270506
2016-07-15 18:56:03brett.cannonsetstatus: open -> closed
resolution: fixed
messages: + msg270500
2016-07-15 18:55:29python-devsetnosy: + python-dev
messages: + msg270499
2016-07-12 05:16:45Brian.Hermansetnosy: + Brian.Herman
messages: + msg270227
2016-05-25 04:18:49serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg266302
2016-05-24 18:07:45brett.cannonsetassignee: brett.cannon
2016-05-22 06:45:32eryksuncreate