It doesn’t always fail, and sometimes only one or the other test fails. Also, it seems to depend on the way the tests are run. I have never seen it fail with “python -m test -v”. Two commands where it does often fail for me:

./python -m test -w test_pkg
./python -m unittest -v test.test_pkg

Here is the output when both tests fail:

test_1 (test.test_pkg.TestPkg) ... ok
test_2 (test.test_pkg.TestPkg) ... ok
test_3 (test.test_pkg.TestPkg) ... ok
test_4 (test.test_pkg.TestPkg) ... ERROR
test_5 (test.test_pkg.TestPkg) ... ok
test_6 (test.test_pkg.TestPkg) ... ok
test_7 (test.test_pkg.TestPkg) ... FAIL
test_8 (test.test_pkg.TestPkg) ... ok

======================================================================
ERROR: test_4 (test.test_pkg.TestPkg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/media/disk/home/proj/python/cpython/Lib/test/test_pkg.py", line 180, in test_4
    self.run_code(s)
  File "/media/disk/home/proj/python/cpython/Lib/test/test_pkg.py", line 69, in run_code
    exec(textwrap.dedent(code), globals(), {"self": self})
  File "<string>", line 2, in <module>
  File "/tmp/tmprhpx80bw/t4.py", line 1, in <module>
RuntimeError: Shouldnt load t4.py

======================================================================
FAIL: test_7 (test.test_pkg.TestPkg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/media/disk/home/proj/python/cpython/Lib/test/test_pkg.py", line 260, in test_7
    '__name__', '__package__', '__path__', '__spec__'])
AssertionError: Lists differ: ['__c[34 chars]__loader__', '__name__', '__package__', '__spec__'] != ['__c[34 chars]__loader__', '__name__', '__package__', '__path__', '__spec__']

First differing element 6:
__spec__
__path__

Second list contains 1 additional elements.
First extra element 7:
__spec__

  ['__cached__',
   '__doc__',
   '__file__',
   '__loader__',
   '__name__',
   '__package__',
+  '__path__',
   '__spec__']

----------------------------------------------------------------------
Ran 8 tests in 0.038s

FAILED (failures=1, errors=1)

Reproduced.  In four runs, one errored in test_4, one failed in test_7, and the other two did both.

redirecting stdout seems to reliably suppress both failures:

$ ./python -m test -w test_pkg >foo
$ cat foo
[1/1] test_pkg
1 test OK.

I think I traced this to revision fbbf8b160e8d (Issue 9548, Python 3.4). Before that revision, I haven’t been able to produce the failures. Also, reverting that change in the current default branch seems to stop the failures. Attached is the patch that reverts the change, though I am not suggesting that is the ultimate fix.

FTR my main setup is x86-64 Arch Linux; I also see this on a 32-bit Linux computer.

I think this was fixed with https://bugs.python.org/issue34200 though the patch was not applied to 3.5 and 3.4 as they are in security bug fixes mode.

Thanks

I am closing this issue as duplicate as part of triaging adding issue34200 as superseder that reports the same error along with a PR and discussion. There have been no failures reported with respect to test_pkg after the fix was merged. Feel free to reopen this if needed.

Thanks