Hi, As far as I see, unittest.TestLoader doesn't search in PEP-420 packages, i.e. packages without __init__.py files.

Is there some motivation behind this, or the loader was just not yet adapted for Implicit Namespace Packages?

Ok, it's actually not so hard to work around (for Python 3.6, at least):

import os
from unittest import TestLoader

class CustomTestLoader(TestLoader):
    def _find_test_path(self, full_path, pattern, namespace=False):
        original_isfile = os.path.isfile

        def patched_isfile(path):
            return str(path).endswith('__init__.py') or original_isfile(path)

        os.path.isfile = patched_isfile
        result = super()._find_test_path(full_path=full_path, pattern=pattern,
                                         namespace=namespace)
        os.path.isfile = original_isfile
        return result

I'll try to submit a pull request if it can be resolved properly.

Alright, I made an initial fix in #282. I believe that I still need
to update the docs and run it with something big, e.g. Django.

Docs promise already support for namespace packages, so just a minor clarification was done.

Ok, testing with Django was a bad idea due to compatibility with 3.7.
I could apply it to 3.6.x and test Django with it.

Is there a better idea how to trial the test discovery?

Testing with Django seems indicated an issue. I did the following with 3.6 patch (cherry-pick to bea9d2f) on macOS with OpenSSL installed via Homebrew:

$ cd /Users/andrei/Python/cpython/
$ export CFLAGS="-I/usr/local/opt/openssl/include"
$ export LDFLAGS="-L/usr/local/opt/openssl/lib"
$ ./configure --with-pydebug --prefix=/Users/andrei/Python/installed/
$ make -j
$ make install

Then in Django (master, b427f0d674):

$ cd /Users/andrei/Python/django/
$ ../../installed/bin/pip3.6 install -r ./requirements/py3.txt
$ PYTHONPATH=.. DJANGO_SETTINGS_MODULE=test_sqlite ../../installed/bin/python3.6 ./runtests.py

That produced one error:

======================================================================
ERROR: auth_tests.test_hashers (unittest.loader._FailedTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/case.py", line 601, in run
    testMethod()
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/loader.py", line 34, in testFailure
    raise self._exception
ImportError: Failed to import test module: auth_tests.test_hashers
Traceback (most recent call last):
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/loader.py", line 426, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Users/andrei/Python/installed/lib/python3.6/unittest/loader.py", line 367, in _get_module_from_name
    __import__(name)
  File "/Users/andrei/Python/django/tests/auth_tests/test_hashers.py", line 20, in <module>
    if crypt.crypt('', '') is None:
  File "/Users/andrei/Python/installed/lib/python3.6/crypt.py", line 47, in crypt
    return _crypt.crypt(word, salt)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfb in position 1: invalid start byte

Ran 11695 tests in 259.390s
FAILED (errors=1, skipped=1149, expected failures=4)

Then I ran the same tests in 3.6.0 virtualenv installed via pyenv:

$ pyenv virtualenv 3.6.0 djtest
$ pyenv shell djtest
$ pip install -r ./requirements/py3.txt
$ PYTHONPATH=.. DJANGO_SETTINGS_MODULE=test_sqlite python ./runtests.py

and they went fine:

Ran 11723 tests in 87.369s
OK (skipped=1149, expected failures=4)

So the patch causes 1 error and misses 28 tests. I'll try to figure out the problem with failing test and what tests are missing.

Removing --with-pydebug parameter helped to avoid issue with _crypto extension. Testing Django with that build produced result identical to 3.6.0:

Ran 11723 tests in 83.897s
OK (skipped=1149, expected failures=4)

The patch is ready for review.

I'm afraid this change makes testloader searches unrelated directory contains massive files (like node_modules).

I don't think loading all tests from whole namespace package is not usual use case.

When using import, (namespace) package name is explicitly specified.
Only specified name is searched.

In test loader's case, there are no such limit.
Loader may search millions of completely unrelated directories.
It may include directories in NFS or samba over slow network.

I was wrong. The ticket can be closed now.