classification
Title: test_pkgutil:test_name_resolution fails on some platforms
Type: Stage:
Components: Build, Distutils, Tests Versions: Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: RJ722, The Compiler, brett.cannon, dstufft, eric.araujo, ned.deily, vinay.sajip
Priority: deferred blocker Keywords:

Created on 2020-06-29 06:56 by RJ722, last changed 2020-06-29 19:46 by brett.cannon.

Messages (5)
msg372549 - (view) Author: Rahul Jha (RJ722) * Date: 2020-06-29 06:56
After configuring and building using the command:

    ./configure --with-pydebug ** make -j

I ran the test suite (without changing anything) and saw that test_pkg has failed.  Here is the output of `./python.exe -m test -v test_pkgutil`:

== CPython 3.10.0a0 (heads/master:cd3c2bdd5d, Jun 28 2020, 13:29:09) [Clang 9.0.0 (clang-900.0.39.2)]
== macOS-10.12.6-x86_64-i386-64bit little-endian
== cwd: /Users/rahuljha/Documents/cpython/build/test_python_10678æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 2.01 Run tests sequentially
0:00:00 load avg: 2.01 [1/1] test_pkgutil
test_getdata_filesys (test.test_pkgutil.PkgutilTests) ... ok
test_getdata_zipfile (test.test_pkgutil.PkgutilTests) ... ok
test_name_resolution (test.test_pkgutil.PkgutilTests) ... ERROR
test_unreadable_dir_on_syspath (test.test_pkgutil.PkgutilTests) ... ok
test_walk_packages_raises_on_string_or_bytes_input (test.test_pkgutil.PkgutilTests) ... ok
test_walkpackages_filesys (test.test_pkgutil.PkgutilTests) ... ok
test_walkpackages_zipfile (test.test_pkgutil.PkgutilTests)
Tests the same as test_walkpackages_filesys, only with a zip file. ... ok
test_alreadyloaded (test.test_pkgutil.PkgutilPEP302Tests) ... ok
test_getdata_pep302 (test.test_pkgutil.PkgutilPEP302Tests) ... ok
test_iter_importers (test.test_pkgutil.ExtendPathTests) ... ok
test_mixed_namespace (test.test_pkgutil.ExtendPathTests) ... ok
test_simple (test.test_pkgutil.ExtendPathTests) ... ok
test_nested (test.test_pkgutil.NestedNamespacePackageTest) ... ok
test_find_loader_avoids_emulation (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_find_loader_missing_module (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_importer_avoids_emulation (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_None_in_sys_modules (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_avoids_emulation (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_handles_missing_loader_attribute (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_handles_missing_spec_attribute (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_handles_spec_attribute_none (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_importer_deprecated (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_iter_importers_avoids_emulation (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_loader_deprecated (test.test_pkgutil.ImportlibMigrationTests) ... ok

======================================================================
ERROR: test_name_resolution (test.test_pkgutil.PkgutilTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rahuljha/Documents/cpython/Lib/test/test_pkgutil.py", line 262, in test_name_resolution
    mod = importlib.import_module(uw)
  File "/Users/rahuljha/Documents/cpython/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'é'

----------------------------------------------------------------------

Ran 24 tests in 0.186s

FAILED (errors=1)
test test_pkgutil failed
test_pkgutil failed

== Tests result: FAILURE ==

1 test failed:
    test_pkgutil

Total duration: 482 ms
Tests result: FAILURE
Py vulture ~/Documents/cpython master !1 ?1 ❯ ./python.exe -m test -v test_pkgutil
== CPython 3.10.0a0 (heads/master:cd3c2bdd5d, Jun 28 2020, 13:29:09) [Clang 9.0.0 (clang-900.0.39.2)]
== macOS-10.12.6-x86_64-i386-64bit little-endian
== cwd: /Users/rahuljha/Documents/cpython/build/test_python_21819æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 2.69 Run tests sequentially
0:00:00 load avg: 2.69 [1/1] test_pkgutil
test_getdata_filesys (test.test_pkgutil.PkgutilTests) ... ok
test_getdata_zipfile (test.test_pkgutil.PkgutilTests) ... ok
test_name_resolution (test.test_pkgutil.PkgutilTests) ... ERROR
test_unreadable_dir_on_syspath (test.test_pkgutil.PkgutilTests) ... ok
test_walk_packages_raises_on_string_or_bytes_input (test.test_pkgutil.PkgutilTests) ... ok
test_walkpackages_filesys (test.test_pkgutil.PkgutilTests) ... ok
test_walkpackages_zipfile (test.test_pkgutil.PkgutilTests)
Tests the same as test_walkpackages_filesys, only with a zip file. ... ok
test_alreadyloaded (test.test_pkgutil.PkgutilPEP302Tests) ... ok
test_getdata_pep302 (test.test_pkgutil.PkgutilPEP302Tests) ... ok
test_iter_importers (test.test_pkgutil.ExtendPathTests) ... ok
test_mixed_namespace (test.test_pkgutil.ExtendPathTests) ... ok
test_simple (test.test_pkgutil.ExtendPathTests) ... ok
test_nested (test.test_pkgutil.NestedNamespacePackageTest) ... ok
test_find_loader_avoids_emulation (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_find_loader_missing_module (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_importer_avoids_emulation (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_None_in_sys_modules (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_avoids_emulation (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_handles_missing_loader_attribute (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_handles_missing_spec_attribute (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_get_loader_handles_spec_attribute_none (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_importer_deprecated (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_iter_importers_avoids_emulation (test.test_pkgutil.ImportlibMigrationTests) ... ok
test_loader_deprecated (test.test_pkgutil.ImportlibMigrationTests) ... ok

======================================================================
ERROR: test_name_resolution (test.test_pkgutil.PkgutilTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rahuljha/Documents/cpython/Lib/test/test_pkgutil.py", line 262, in test_name_resolution
    mod = importlib.import_module(uw)
  File "/Users/rahuljha/Documents/cpython/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'é'

----------------------------------------------------------------------

Ran 24 tests in 0.184s

FAILED (errors=1)
test test_pkgutil failed
test_pkgutil failed

== Tests result: FAILURE ==

1 test failed:
    test_pkgutil

Total duration: 635 ms
Tests result: FAILURE
msg372550 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-06-29 07:24
This same failure is observed with the most recent 3.9beta (3.9.0b3) when running on macOS 10.9 with the binaries from the python.org macOS installer.  However, no failure is seen when running the same Python on macOS 10.15. I have not investigated further with other releases.  This test failure should be addressed prior t0 3.9.0rc1.
msg372551 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-06-29 07:25
PS, the problem may have been introduced by the changes for bpo-12915.
msg372603 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-06-29 17:06
The problem appears to be in importlib. I put in a breakpoint in the test just before it tries to import the module é. (At this point it has successfully imported a module called वमस in an earlier loop iteration.)

Then, in a different terminal console, I cd'd to the test directory and tried the import directly:
$ ls -l
total 0
drwxr-xr-x  3 xxxxxxxx  xxxxx  102 29 Jun 17:55 é
drwxr-xr-x  4 xxxxxxxx  xxxxx  136 29 Jun 17:55 वमस
$ ~/projects/cpython/python.exe
Python 3.10.0a0 (heads/master:a3ad95d, Jun 29 2020, 17:50:09)
[Clang 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import वमस
>>> import é
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'é'

So for some reason, importlib is failing to import the é module. (The é/__init__.py is a zero-byte file.)

Adding Brett to nosy.
msg372624 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-06-29 19:46
I wonder if it's a problem from Unicode representation since I don't think importlib tries to have a standard normalization form of Unicode. I could see that leading to how the file system encoded something differing from how it was encoded in the test file, leading to a claim the file doesn't exist.

And thanks to binary file paths that isn't necessarily a wrong result. :P
History
Date User Action Args
2020-06-29 19:46:05brett.cannonsetmessages: + msg372624
2020-06-29 17:06:04vinay.sajipsetnosy: + brett.cannon
messages: + msg372603
2020-06-29 10:38:57vstinnersetnosy: - vstinner
2020-06-29 07:34:25The Compilersetnosy: + The Compiler
2020-06-29 07:25:37ned.deilysetmessages: + msg372551
2020-06-29 07:24:18ned.deilysetpriority: normal -> deferred blocker
title: test_pkgutil:test_name_resolution fails on master -> test_pkgutil:test_name_resolution fails on some platforms
nosy: + ned.deily, vstinner

messages: + msg372550

versions: + Python 3.9
2020-06-29 06:56:41RJ722create