This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: MSVCCompiler creates redundant and long PATH strings
Type: Stage:
Components: Distutils Versions: Python 2.5
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: nnorwitz, scott.dial
Priority: normal Keywords: patch

Created on 2007-03-21 22:05 by scott.dial, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
msvccompiler.py.diff scott.dial, 2007-03-21 22:05 Patch for Lib/distutils/msvccompiler.py
Messages (3)
msg52277 - (view) Author: Scott Dial (scott.dial) Date: 2007-03-21 22:05
The current code in MSVCCompiler.initialiaze will throw  an OSError on the assignment to os.environ['path'] after repeated usages of MSVCCompilers because the string produced may exceeed 4096 characters (usually after several compilations). In the call to initialize, the needed paths for compiling via the compiler selected are added on to the front of the path regardless of what the current path is set to be. This is bad because we could be adding redundant paths (and thus exceeding the 4096 character limit needlessly)

Furthmore, this happens regularly whenever anyone uses the VCToolkit2003 compiler because you end up in the "if self.__version >= 7:" control path of get_msvc_paths which merely returns the current path back to the callee. This results in initialize /duplicating the entire path/.

The proposed patch adds a pass through the new path that removes duplicates while maintaining the ordering.

Unfortunately it is difficult for me to provide a real test case for this as it arises out of trying to run the translate code with PyPy. However, I hope that the change is obviously fixing a bug that I've made clear exists.
msg52278 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-04-01 18:30
Committed revision 54644.
Committed revision 54645. (2.5)

Backporting might be considered a little sketchy, but it fixes a real problem and is not too likely to break something important (ha! that's just inviting Murphy in).  Please test 2.5.1c1 that will be coming out in a few days.  I modified the patch to use a utility function and added comments and docstrings.
msg52279 - (view) Author: Scott Dial (scott.dial) Date: 2007-04-01 18:58
The revised patch looks good. I think the comment about it being O(n**2) is not to be worrisome. I suppose you could argue for something like:

    return list(set([os.path.normpath(p) for p in paths]))

However, you would lose the ordering, which must be maintained. At worst, the path can only be 4096 characters (at most 1024 entries of "C:\\"). Except in the face of what will be an error (the final path is going to be too large), we are bounded from above at n=2048, which is plenty fast.

I think you probably guess that much, but at least my rationale for such a simple algorithm is recorded.
History
Date User Action Args
2022-04-11 14:56:23adminsetgithub: 44754
2007-03-21 22:05:30scott.dialcreate