http://buildbot.python.org/all/builders/AMD64%20Debian%20root%202.7/builds/154/steps/test/logs/stdio

0:04:59 [312/402/1] test_thread failed
Traceback (most recent call last):
  File "/root/buildarea/2.7.angelico-debian-amd64/build/Lib/test/test_thread.py", line 133, in task
    raise SyntaxError
SyntaxError: None
test test_thread failed -- Traceback (most recent call last):
  File "/root/buildarea/2.7.angelico-debian-amd64/build/Lib/test/test_thread.py", line 150, in test_save_exception_state_on_error
    self.assertIn("Traceback", stderr.getvalue())
AssertionError: 'Traceback' not found in 'Unhandled exception in thread started by <function task at 0x7f06c77ab1b0>\n'

I just encountered with this failure. It is random, I can reproduce it only when run test_thread repeatedly in parallel with running other tests (maybe needed high load factor).

$ ./python -m test -uall -F test_thread test_thread test_thread test_thread 
Run tests sequentially
0:00:00 [  1] test_thread
0:00:00 [  2] test_thread
Traceback (most recent call last):
  File "/home/serhiy/py/cpython2.7/Lib/test/test_thread.py", line 133, in task
    raise SyntaxError
SyntaxError: None
test test_thread failed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython2.7/Lib/test/test_thread.py", line 150, in test_save_exception_state_on_error
    self.assertIn("Traceback", stderr.getvalue())
AssertionError: 'Traceback' not found in 'Unhandled exception in thread started by <function task at 0xb6a89ee4>\n'

1 test OK.
1 test failed:
test_thread

Total duration: 432 ms
Tests result: FAILURE
[50664 refs]

Problem is with test test_thread.ThreadRunningTests.test_save_exception_state_on_error when other tests leave threads runnig.

test_save_exception_state_on_error relay on thread._get_count(), if this value decrease test assume thread is finished with is not always correct (other threads finish - started by different test).

Fix is to make sure each test wait for all threads to finsh.

The solution in PR 1583 looks unsafe to me. If some threads hang the loop while self.thread_count != thread._count() will never finished. If some dangling threads created in other tests or by regrtest itself are finished during running the test_thread test, that loop will never finished too.

Using 5 terminals to run 5 tests in parallel.

I'm unable to reproduce the bug if I only run the test alone:

./python -m test -uall -F -m test_save_exception_state_on_error test_thread

But I'm able to reproduce the bug if I run the full test_thread.py:

./python -m test -uall -F test_thread

So I'm now more confident that grzgrzgrz3's patch can fix the issue. test_thread uses the low-level thread.start_new_thread() function to spawn threads, but pthread_join() is not used to wait for the thread exit. So multiple test_thread tests can "leak" threads which can have random effect on following tests.

New changeset 79ef7f8 by Victor Stinner in branch 'master':
bpo-30357: test_thread now uses threading_cleanup() (bpo-1592)
79ef7f8

I think this do not solve this issue yet. There is still posibillity that different tests/testrunners spawn threads and 'fool' testcase. I think we should not relay on thread._count value where it's possible.

For master branch thread._set_sentinel() can be used. For 2.7 i don't know simple solution.

I would like to know your opinion it is worth changing?

There is still posibillity that different tests/testrunners spawn threads
and 'fool' testcase.

If you run multiple test files in parallel, each test runner process runs a
single test at the same time: test methods are run sequentially to prevent
side effects.

New changeset 6924ed5 by Victor Stinner (grzgrzgrz3) in branch '2.7':
bpo-30357 each test in test_thread waits until all spawned threads finish (bpo-1583)
6924ed5

Backported to 3.5 (aeb6447) and 3.6 (6b5b85a).

I now close the issue, I consider that the change should fix the randomly failing test. If the failure comes back, I will reopen the issue.

Thank you very much Grzegorz Grzywacz for your first contribution!

New changeset f8d05b3 by Victor Stinner in branch 'master':
bpo-30387: Fix warning in test_threading (bpo-1634)
f8d05b3

New changeset 44944b6 by Victor Stinner in branch '3.6':
bpo-30387: Fix warning in test_threading (bpo-1634) (bpo-1636)
44944b6

New changeset f5633e0 by Victor Stinner in branch '3.5':
bpo-30387: Fix warning in test_threading (bpo-1634) (bpo-1637)
f5633e0