$ ./python Lib/test/test_runpy.py
.......................== CPython 3.7.0a0 (heads/master-dirty:39243779f4, Jul 25 2017, 14:32:21) [GCC 6.3.0 20170406]
== Linux-4.10.0-28-generic-i686-athlon-with-debian-stretch-sid little-endian
== hash algorithm: siphash24 32bit
== cwd: /home/serhiy/py/cpython/build/test_python_31873
== CPU count: 2
== encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 load avg: 0.85 [  1/406] test_grammar
0:00:00 load avg: 0.85 [  2/406] test_opcodes
0:00:00 load avg: 0.85 [  3/406] test_dict
0:00:01 load avg: 0.85 [  4/406] test_builtin
0:00:02 load avg: 0.85 [  5/406] test_exceptions
0:00:02 load avg: 0.85 [  6/406] test_types
...

In isolated mode it runs normally.

$ ./python -I Lib/test/test_runpy.py
..............................
----------------------------------------------------------------------
Ran 30 tests in 2.776s

OK

After commenting out methods test_directory_error, test_zipfile_error and test_main_recursion_error, test_runpy is passed.

This is a result of a known quirk in the way sys.path entry execution works: the search for "__main__.py" isn't constrained specifically to sys.path[0].

That's almost entirely a bad thing, but I'd been ignoring it because I hadn't thought of a nice way of fixing it that didn't require some substantial changes to the import system APIs.

However, it just occurred to me that I may have been overcomplicating matters: we don't need to keep runpy from *finding* a __main__.py from outside sys.path[0] in this case, we just need to keep it from *running* it.

That means that after we find the candidate module spec for __main__, we can introduce a new constraint:

    if os.path.commonpath([sys.path[0], spec.origin]) != sys.path[0]:
        raise RuntimeError(...)

It might still be a little fiddly to decide exactly when to enforce the constraint, but it should still be much easier than attempting to constrain the search for the spec directly.

Reproduced on 3.11.