classification
Title: test_import: test_unwritable_module() fails on AMD64 Fedora Stable Clang Installed 3.x
Type: Stage: resolved
Components: Tests Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, dino.viehland, vstinner
Priority: normal Keywords: patch

Created on 2020-01-27 09:19 by vstinner, last changed 2020-01-28 22:16 by dino.viehland. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 18203 merged vstinner, 2020-01-27 16:39
PR 18211 merged dino.viehland, 2020-01-27 20:30
PR 18241 merged dino.viehland, 2020-01-28 18:52
Messages (18)
msg360743 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-27 09:19
AMD64 Fedora Stable Clang Installed 3.x:
https://buildbot.python.org/all/#/builders/127/builds/212

test_unwritable_module (test.test_import.CircularImportTests) ... ERROR

======================================================================
ERROR: test_unwritable_module (test.test_import.CircularImportTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/test_import/__init__.py", line 1347, in test_unwritable_module
    import test.test_import.data.unwritable as unwritable
ModuleNotFoundError: No module named 'test.test_import.data.unwritable'
msg360744 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-27 09:19
Test added in bpo-39336 by:

commit 9b6fec46513006d7b06fcb645cca6e4f5bf7c7b8
Author: Dino Viehland <dinoviehland@gmail.com>
Date:   Wed Jan 22 16:42:38 2020 -0800

    bpo-39336: Allow packages to not let their child modules be set on them (#18006)
    
    * bpo-39336: Allow setattr to fail on modules which aren't assignable
    
    When attaching a child module to a package if the object in sys.modules raises an Attribu
teError (e.g. because it is immutable) it causes the whole import to fail.  This now allows i
mmutable packages to exist and an ImportWarning is reported and the AttributeError exception 
is ignored.
msg360746 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-27 09:20
Same error on x86 Gentoo Installed with X 3.x:
https://buildbot.python.org/all/#/builders/128/builds/204
msg360776 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-27 16:31
Issue also seen when building Python 3.9.0a3 on Fedora Rawhide:
https://src.fedoraproject.org/rpms/python39/pull-request/16

Logs: https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-rawhide-pr-pipeline/2558/artifact/package-tests/logs/console.log

pythoninfo:

    CC.version: gcc (GCC) 10.0.1 20200126 (Red Hat 10.0.1-0.6)
    Py_DEBUG: No (sys.gettotalrefcount() missing)
    config[executable]: '/usr/bin/python3.9'
    config[module_search_paths]: ['/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload']
    os.getcwd: /var/str/python/selftest
    os.getgid: 0
    os.getgrouplist: 0
    os.getgroups: 0
    os.getuid: 0
    os.login: root
    platform.architecture: 64bit ELF
    platform.libc_ver: glibc 2.30.9000
    platform.platform: Linux-5.5.0-0.rc6.git3.1.fc32.x86_64-x86_64-with-glibc2.30.9000
    platform.python_implementation: CPython
    pwd.getpwuid(0): pwd.struct_passwd(pw_name='root', pw_passwd='x', pw_uid=0, pw_gid=0, pw_gecos='root', pw_dir='/root', pw_shell='/bin/bash')
msg360778 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-27 16:41
> https://src.fedoraproject.org/rpms/python39/pull-request/16
>     os.getuid: 0

Maybe this failure comes from the fact that the test suite was run as the root user. I wrote PR 18203 to log the effective uid and gid in pythoninfo, to learn more about this issue.
msg360781 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-27 17:06
New changeset 4a46adc7746930c4589ee483cad88d3f8504c045 by Victor Stinner in branch 'master':
bpo-39459: test.pythoninfo logs effective uid/gid (GH-18203)
https://github.com/python/cpython/commit/4a46adc7746930c4589ee483cad88d3f8504c045
msg360789 - (view) Author: Dino Viehland (dino.viehland) * (Python committer) Date: 2020-01-27 19:30
The curious thing about this is other tests in CircularImportTests are importing packages from test.test_import.data in the exact same way.
msg360792 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-27 19:58
> AMD64 Fedora Stable Clang Installed 3.x:
> https://buildbot.python.org/all/#/builders/127/builds/212

So, I added effective uid/gid to pythoninfo, here you have:

os.getegid: 1000
os.geteuid: 1000
os.getgid: 1000
os.getgrouplist: 1000
os.getgroups: 1000
os.getresgid: (1000, 1000, 1000)
os.getresuid: (1000, 1000, 1000)
os.getuid: 1000

There is nothing related to root (uid 0).
msg360796 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-01-27 20:16
Was the Makefile updated to install Lib/test/test_import/date/unwritable?
msg360797 - (view) Author: Dino Viehland (dino.viehland) * (Python committer) Date: 2020-01-27 20:25
Ahh, that's probably it Brett, I didn't know that was there, thanks!
msg360820 - (view) Author: Dino Viehland (dino.viehland) * (Python committer) Date: 2020-01-27 23:08
I've added the files to the makefile and AMD64 Fedora Stable Clang Installed 3.x was passing.
msg360831 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-28 02:20
commit 2528a6c3d0660c03ae43d796628462ccf8e58190
Author: Dino Viehland <dinoviehland@gmail.com>
Date:   Mon Jan 27 14:04:56 2020 -0800

    Add test.test_import.data.unwritable package to makefile (#18211)
msg360833 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-28 02:24
> I've added the files to the makefile and AMD64 Fedora Stable Clang Installed 3.x was passing.

Thanks for the fix! I didn't notice that only "Installed" buildbot workers were impacted.
msg360891 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-01-28 18:37
Did the Windows-equivalent project files also get updated?
msg360902 - (view) Author: Dino Viehland (dino.viehland) * (Python committer) Date: 2020-01-28 21:24
New changeset 0cd5bff6b7da3118d0c5a88fc2b80f80eb7c3059 by Dino Viehland in branch 'master':
bpo-39459: include missing test files in windows installer 
https://github.com/python/cpython/commit/0cd5bff6b7da3118d0c5a88fc2b80f80eb7c3059
msg360903 - (view) Author: Dino Viehland (dino.viehland) * (Python committer) Date: 2020-01-28 21:24
Nope, thank you for pointing that out.  I've updated them now with PR 18241
msg360904 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-28 21:57
The Windows installer copies Lib/test/ and subdirectories: see <InstallFiles Include="$(PySourcePath)Lib\test\**\*" ...> in Tools/msi/test/test.wixproj.
msg360907 - (view) Author: Dino Viehland (dino.viehland) * (Python committer) Date: 2020-01-28 22:16
I guess the update to lib.pyproj probably just makes the files show up when opening the solution in Visual Studio then.
History
Date User Action Args
2020-01-28 22:16:12dino.viehlandsetmessages: + msg360907
2020-01-28 21:57:43vstinnersetmessages: + msg360904
2020-01-28 21:24:59dino.viehlandsetmessages: + msg360903
2020-01-28 21:24:37dino.viehlandsetmessages: + msg360902
2020-01-28 18:52:17dino.viehlandsetpull_requests: + pull_request17621
2020-01-28 18:37:03brett.cannonsetmessages: + msg360891
2020-01-28 02:24:12vstinnersetmessages: + msg360833
2020-01-28 02:20:56vstinnersetmessages: + msg360831
2020-01-27 23:08:32dino.viehlandsetstatus: open -> closed
resolution: fixed
messages: + msg360820

stage: patch review -> resolved
2020-01-27 20:30:05dino.viehlandsetpull_requests: + pull_request17590
2020-01-27 20:25:54dino.viehlandsetmessages: + msg360797
2020-01-27 20:16:10brett.cannonsetnosy: + brett.cannon
messages: + msg360796
2020-01-27 19:58:51vstinnersetmessages: + msg360792
2020-01-27 19:30:30dino.viehlandsetmessages: + msg360789
2020-01-27 17:06:45vstinnersetmessages: + msg360781
2020-01-27 16:41:56vstinnersetmessages: + msg360778
2020-01-27 16:39:00vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request17580
2020-01-27 16:31:24vstinnersetmessages: + msg360776
2020-01-27 09:20:50vstinnersetmessages: + msg360746
2020-01-27 09:20:10vstinnersetnosy: + dino.viehland
2020-01-27 09:19:25vstinnersetmessages: + msg360744
2020-01-27 09:19:04vstinnercreate