Title: Why does still rely on existence of files?
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.7, Python 3.6, Python 3.5
Status: closed Resolution: not a bug
Dependencies: Superseder: unittest discovery doesn't detect namespace packages when given no parameters
View: 23882
Assigned To: Nosy List: Andrei Fokau, Andrei Fokau2, barry, ezio.melotti, methane, michael.foord, nedbat, rbcollins
Priority: normal Keywords:

Created on 2017-02-24 16:48 by Andrei Fokau, last changed 2020-07-19 03:35 by methane. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 282 closed python-dev, 2017-02-24 21:56
Messages (10)
msg288529 - (view) Author: Andrei Fokau (Andrei Fokau) * Date: 2017-02-24 16:48
Hi, As far as I see, unittest.TestLoader doesn't search in PEP-420 packages, i.e. packages without files.

Is there some motivation behind this, or the loader was just not yet adapted for Implicit Namespace Packages?
msg288531 - (view) Author: Andrei Fokau (Andrei Fokau) * Date: 2017-02-24 20:16
Ok, it's actually not so hard to work around (for Python 3.6, at least):

import os
from unittest import TestLoader

class CustomTestLoader(TestLoader):
    def _find_test_path(self, full_path, pattern, namespace=False):
        original_isfile = os.path.isfile

        def patched_isfile(path):
            return str(path).endswith('') or original_isfile(path)

        os.path.isfile = patched_isfile
        result = super()._find_test_path(full_path=full_path, pattern=pattern,
        os.path.isfile = original_isfile
        return result

I'll try to submit a pull request if it can be resolved properly.
msg288534 - (view) Author: Andrei Fokau (Andrei Fokau) * Date: 2017-02-24 21:59
Alright, I made an initial fix in #282. I believe that I still need 
to update the docs and run it with something big, e.g. Django.
msg288545 - (view) Author: Andrei Fokau (Andrei Fokau) * Date: 2017-02-25 07:32
Docs promise already support for namespace packages, so just a minor clarification was done.
msg288558 - (view) Author: Andrei Fokau (Andrei Fokau) * Date: 2017-02-25 10:48
Ok, testing with Django was a bad idea due to compatibility with 3.7. 
I could apply it to 3.6.x and test Django with it. 

Is there a better idea how to trial the test discovery?
msg288567 - (view) Author: Andrei Fokau (Andrei Fokau) * Date: 2017-02-25 15:00
Testing with Django seems indicated an issue. I did the following with 3.6 patch (cherry-pick to bea9d2f64) on macOS with OpenSSL installed via Homebrew:

$ cd /Users/andrei/Python/cpython/
$ export CFLAGS="-I/usr/local/opt/openssl/include"
$ export LDFLAGS="-L/usr/local/opt/openssl/lib"
$ ./configure --with-pydebug --prefix=/Users/andrei/Python/installed/
$ make -j
$ make install

Then in Django (master, b427f0d674):

$ cd /Users/andrei/Python/django/
$ ../../installed/bin/pip3.6 install -r ./requirements/py3.txt
$ PYTHONPATH=.. DJANGO_SETTINGS_MODULE=test_sqlite ../../installed/bin/python3.6 ./

That produced one error:

ERROR: auth_tests.test_hashers (unittest.loader._FailedTest)
Traceback (most recent call last):
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/", line 59, in testPartExecutor
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/", line 601, in run
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/", line 34, in testFailure
    raise self._exception
ImportError: Failed to import test module: auth_tests.test_hashers
Traceback (most recent call last):
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/", line 426, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/", line 367, in _get_module_from_name
  File "/Users/andrei/Python/django/tests/auth_tests/", line 20, in <module>
    if crypt.crypt('', '') is None:
  File "/Users/andrei/Python/installed/lib/python3.6/", line 47, in crypt
    return _crypt.crypt(word, salt)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfb in position 1: invalid start byte

Ran 11695 tests in 259.390s
FAILED (errors=1, skipped=1149, expected failures=4)

Then I ran the same tests in 3.6.0 virtualenv installed via pyenv:

$ pyenv virtualenv 3.6.0 djtest
$ pyenv shell djtest
$ pip install -r ./requirements/py3.txt
$ PYTHONPATH=.. DJANGO_SETTINGS_MODULE=test_sqlite python ./

and they went fine:

Ran 11723 tests in 87.369s
OK (skipped=1149, expected failures=4)

So the patch causes 1 error and misses 28 tests. I'll try to figure out the problem with failing test and what tests are missing.
msg288576 - (view) Author: Andrei Fokau (Andrei Fokau) * Date: 2017-02-25 19:35
Removing `--with-pydebug` parameter helped to avoid issue with _crypto extension. Testing Django with that build produced result identical to 3.6.0:

Ran 11723 tests in 83.897s
OK (skipped=1149, expected failures=4)

The patch is ready for review.
msg288697 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2017-02-28 07:45
I'm afraid this change makes testloader searches unrelated directory contains massive files (like node_modules).

I don't think loading all tests from whole namespace package is not usual use case.
msg288698 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2017-02-28 07:51
When using import, (namespace) package name is explicitly specified.
Only specified name is searched.

In test loader's case, there are no such limit.
Loader may search millions of completely unrelated directories.
It may include directories in NFS or samba over slow network.
msg290064 - (view) Author: Andrei Fokau (Andrei Fokau2) Date: 2017-03-23 22:10
I was wrong. The ticket can be closed now.
Date User Action Args
2020-07-19 03:35:55methanesetsuperseder: unittest discovery doesn't detect namespace packages when given no parameters
2017-04-12 04:02:31berker.peksagsetstatus: open -> closed
resolution: not a bug
stage: patch review -> resolved
2017-03-23 23:05:52nedbatsetnosy: + nedbat
2017-03-23 22:10:40Andrei Fokau2setnosy: + Andrei Fokau2
messages: + msg290064
2017-02-28 07:51:39methanesetmessages: + msg288698
2017-02-28 07:45:00methanesetnosy: + methane
messages: + msg288697
2017-02-25 19:35:56Andrei Fokausetmessages: + msg288576
2017-02-25 15:00:45Andrei Fokausetmessages: + msg288567
2017-02-25 10:48:17Andrei Fokausetmessages: + msg288558
2017-02-25 07:32:43Andrei Fokausetmessages: + msg288545
2017-02-25 01:50:07Mariattasetversions: - Python 3.4
2017-02-24 22:19:37serhiy.storchakasetnosy: + rbcollins, ezio.melotti, michael.foord

stage: patch review
2017-02-24 21:59:13Andrei Fokausetmessages: + msg288534
2017-02-24 21:56:07python-devsetpull_requests: + pull_request251
2017-02-24 20:17:00Andrei Fokausetmessages: + msg288531
2017-02-24 16:52:51barrysetnosy: + barry
2017-02-24 16:48:45Andrei Fokaucreate