classification
Title: unittest can unnecessarily modify sys.path (and with the wrong case)
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.4, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: cdyson37, chris.jerdonek, michael.foord, xtreak
Priority: normal Keywords:

Created on 2014-12-21 08:42 by chris.jerdonek, last changed 2018-10-02 09:27 by cdyson37.

Messages (2)
msg232993 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2014-12-21 08:42
I have observed that when running unit tests using unittest's test discovery, unittest can simultaneously (1) modify sys.path unnecessarily (by adding a path that is already in sys.path with a different case), and (2) modify sys.path by adding a path of the "wrong" case.

This occurs for me on Mac OS X with the default case-insensitive file system.

If the path--

'/Users/chris/Dev/python/my_package'

is already in sys.path, running unittest's test discovery will prepend sys.path with the following:

'/Users/chris/dev/python/my_package'

Aside from causing unnecessary modifications of sys.path, this also causes an issue in coverage, for example:

https://bitbucket.org/ned/coveragepy/issue/348

The relevant code is here in unittest/loader.py (with `top_level_dir` starting out as os.curdir):

    top_level_dir = os.path.abspath(top_level_dir)

    if not top_level_dir in sys.path:
        # all test modules must be importable from the top level directory
        # should we *unconditionally* put the start directory in first
        # in sys.path to minimise likelihood of conflicts between installed
        # modules and development versions?
        sys.path.insert(0, top_level_dir)
    self._top_level_dir = top_level_dir

(from https://hg.python.org/cpython/file/75ede5bec8db/Lib/unittest/loader.py#l259 )

The issue occurs when os.path.abspath(top_level_dir) is already in sys.path but with a different case.  (Note that if os.path.abspath() returned a path with the "right" case, then the unittest code would be okay.)

See also these two threads regarding obtaining the correct case for a path:

1. https://mail.python.org/pipermail/python-dev/2010-September/103823.html
2. https://mail.python.org/pipermail/python-ideas/2010-September/008153.html
msg326870 - (view) Author: Charlie Dyson (cdyson37) Date: 2018-10-02 09:27
As an aside, should that be sys.path.insert(1, X)? As 0 has a special meaning (I've often thought this is a slightly odd convention).

Another aside: I noticed this because I was looking to write a module finder, and thought I could extract one out of this function. It would be nice to have a module-crawler function, and write discover() in terms of that.
History
Date User Action Args
2018-10-02 09:27:03cdyson37setnosy: + cdyson37
messages: + msg326870
2018-09-22 15:50:01xtreaksetnosy: + xtreak
2015-03-21 19:25:42BreamoreBoysetnosy: + michael.foord
2014-12-21 08:42:19chris.jerdonekcreate