classification
Title: trace module compares directories as strings (--ignore-dir)
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder: Improvements to trace._Ignore
View: 10908
Assigned To: Nosy List: SilentGhost, belopolsky, vrutsky
Priority: normal Keywords: patch

Created on 2011-01-12 14:44 by vrutsky, last changed 2011-01-19 15:40 by SilentGhost.

Files
File name Uploaded Description Edit
test.py vrutsky, 2011-01-12 14:44 test Python file
test.cmd vrutsky, 2011-01-12 14:45 shell script to run test
test.out vrutsky, 2011-01-12 14:45 output for shell script
trace.diff SilentGhost, 2011-01-12 17:03 review
trace_ignore_case_fix.patch vrutsky, 2011-01-19 15:10 Fix case for --ignore-dir and __file__ review
Messages (7)
msg126101 - (view) Author: Vladimir Rutsky (vrutsky) * Date: 2011-01-12 14:44
This is code from recent trace.py (http://svn.python.org/view/python/branches/release27-maint/Lib/trace.py?view=markup):

trace.py:169:
        # Ignore a file when it contains one of the ignorable paths
        for d in self._dirs:
            # The '+ os.sep' is to ensure that d is a parent directory,
            # as compared to cases like:
            #  d = "/usr/local"
            #  filename = "/usr/local.py"
            # or
            #  d = "/usr/local.py"
            #  filename = "/usr/local.py"
            if filename.startswith(d + os.sep):
                self._ignore[modulename] = 1
                return 1

Directories comparison like "filename.startswith(d + os.sep)" works incorrect on case-insensitive filesystems (such as NTFS on Windows).

This leads to confusing results on Windows:
  python trace.py --ignore-dir='$prefix' -t test.py
or
  python trace.py --ignore-dir C:\Python26\Lib -t test.py
shows all calls from standard library, but this one doesn't:
  python trace.py --ignore-dir=c:\python26\lib -t test.py

See attached test files and log for details.
msg126110 - (view) Author: Vladimir Rutsky (vrutsky) * Date: 2011-01-12 16:27
Workaround for people on Windows who don't wan't to modify trace.py:
to get clean trace of only your project calls add to ignore list python path with mix of character cases. For me worked out this string:
  python -m trace --ignore-dir=c:\python26\lib;C:\python26\lib;C:\Python26\Lib -t main.py
msg126112 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2011-01-12 17:03
It's due to the os.path.normpath not normalizing case. Here is the patch.

Also affects 3.x
msg126272 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2011-01-14 15:55
#10908 is dealing with this and other issue re ignored dirs.
msg126524 - (view) Author: Vladimir Rutsky (vrutsky) * Date: 2011-01-19 13:56
SilentGhost, thanks for the patch! I can confirm, that adding os.path.normcase fixes issues with different cases of files.

I believe there also may be issue with FAT's long file name mangling, like "C:\PROGRA~1\python26\" instead of "C:\Program Files\python26\". But never experienced such issue with Python.
msg126526 - (view) Author: Vladimir Rutsky (vrutsky) * Date: 2011-01-19 15:10
Sorry I was wrong - patch don't fix original issue.

Case should be normalized not only for directories provided through --ignore-dir, but also for directories obtained from __file__. In my tests on Windows Ignore.names(self, filename, modulename) receives `filename''s argument like:
  C:\Python26\Lib\site-packages\shapely\__init__.pyc

Attaching patch fixes this issue for me (for Python 2.7 from http://svn.python.org/projects/python/branches/release27-maint, patch attached by SilentGhost is for 3.X I think).
msg126535 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2011-01-19 15:40
Vladimir, superseder's patch fixes that too.
History
Date User Action Args
2016-03-23 08:53:22SilentGhostunlinkissue10908 dependencies
2011-01-19 15:40:45SilentGhostsetnosy: belopolsky, SilentGhost, vrutsky
messages: + msg126535
2011-01-19 15:10:11vrutskysetfiles: + trace_ignore_case_fix.patch
nosy: + SilentGhost
messages: + msg126526

2011-01-19 13:56:30vrutskysetnosy: belopolsky, vrutsky
messages: + msg126524
2011-01-15 16:26:12SilentGhostlinkissue10908 dependencies
2011-01-15 16:21:43SilentGhostsetnosy: - SilentGhost
2011-01-15 16:20:45SilentGhostsetstatus: closed -> open
nosy: belopolsky, SilentGhost, vrutsky
2011-01-14 15:55:20SilentGhostsetstatus: open -> closed

superseder: Improvements to trace._Ignore
messages: + msg126272
nosy: belopolsky, SilentGhost, vrutsky
2011-01-12 17:25:29pitrousetnosy: belopolsky, SilentGhost, vrutsky
type: behavior
stage: patch review
2011-01-12 17:23:01SilentGhostsetnosy: + belopolsky
2011-01-12 17:03:17SilentGhostsetfiles: + trace.diff
versions: + Python 3.1, Python 3.2
nosy: + SilentGhost

messages: + msg126112

keywords: + patch
2011-01-12 16:27:19vrutskysetmessages: + msg126110
2011-01-12 14:45:39vrutskysetfiles: + test.out
2011-01-12 14:45:21vrutskysetfiles: + test.cmd
2011-01-12 14:44:16vrutskycreate