classification
Title: test_concurrent_futures crashed with python.core core dump on AMD64 FreeBSD Shared 3.x
Type: Stage: resolved
Components: Tests Versions: Python 3.9
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process
View: 39877
Assigned To: Nosy List: aeros, vstinner
Priority: normal Keywords:

Created on 2019-12-18 17:37 by vstinner, last changed 2020-03-06 14:51 by vstinner. This issue is now closed.

Messages (3)
msg358634 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-12-18 17:37
Yesterday and today, I pushed two test_concurrent_futures fixes in bpo-38546:

* commit 673c39331f844a80c465efd7cff88ac55c432bfb
* commit 9707e8e22d80ca97bf7a9812816701cecde6d226

Maybe it fixed this crash, maybe not.

https://buildbot.python.org/all/#/builders/152/builds/70

0:19:28 load avg: 4.96 [242/420/1] test_concurrent_futures failed (env changed) (5 min 4 sec) -- running: test_io (2 min 1 sec), test_largefile (31.3 sec)
test_cancel (test.test_concurrent_futures.FutureTests) ... ok
test_cancelled (test.test_concurrent_futures.FutureTests) ... ok
test_done (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_already_cancelled (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_already_failed (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_already_successful (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_raises (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_raises_already_succeeded (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_with_cancel (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_with_exception (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_with_result (test.test_concurrent_futures.FutureTests) ... ok
test_exception_with_success (test.test_concurrent_futures.FutureTests) ... ok
test_exception_with_timeout (test.test_concurrent_futures.FutureTests) ... ok
test_multiple_set_exception (test.test_concurrent_futures.FutureTests) ... ok
test_multiple_set_result (test.test_concurrent_futures.FutureTests) ... ok
test_repr (test.test_concurrent_futures.FutureTests) ... ok
test_result_with_cancel (test.test_concurrent_futures.FutureTests) ... ok
test_result_with_success (test.test_concurrent_futures.FutureTests) ... ok
test_result_with_timeout (test.test_concurrent_futures.FutureTests) ... ok
test_running (test.test_concurrent_futures.FutureTests) ... ok
test_correct_timeout_exception_msg (test.test_concurrent_futures.ProcessPoolForkAsCompletedTest) ... 0.35s ok
test_duplicate_futures (test.test_concurrent_futures.ProcessPoolForkAsCompletedTest) ... 2.40s ok
test_free_reference_yielded_future (test.test_concurrent_futures.ProcessPoolForkAsCompletedTest) ... 0.37s ok
test_no_timeout (test.test_concurrent_futures.ProcessPoolForkAsCompletedTest) ... 0.24s ok
test_zero_timeout (test.test_concurrent_futures.ProcessPoolForkAsCompletedTest) ... 2.40s ok
test_crash (test.test_concurrent_futures.ProcessPoolForkExecutorDeadlockTest) ... 1.12s ok
test_shutdown_deadlock (test.test_concurrent_futures.ProcessPoolForkExecutorDeadlockTest) ... 0.57s ok
test_initializer (test.test_concurrent_futures.ProcessPoolForkFailingInitializerTest) ... 0.19s ok
test_initializer (test.test_concurrent_futures.ProcessPoolForkInitializerTest) ... 0.26s ok
test_free_reference (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.50s ok
test_killed_child (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.25s ok
test_map (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.37s ok
test_map_chunksize (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.40s ok
test_map_exception (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.58s ok
test_map_timeout (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 6.42s ok
test_max_workers_negative (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.32s ok
test_max_workers_too_large (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... skipped 'Windows-only process limit'
test_no_stale_references (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.31s ok
test_ressources_gced_in_workers (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.86s ok
test_shutdown_race_issue12456 (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.69s ok
test_submit (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.51s ok
test_submit_keyword (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.16s ok
test_traceback (test.test_concurrent_futures.ProcessPoolForkProcessPoolExecutorTest) ... 0.18s ok
test_context_manager_shutdown (test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest) ... 0.06s ok
test_del_shutdown (test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest) ... 0.07s ok
test_hang_issue12364 (test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest) ... 1.22s ok
test_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest) ... 2.13s ok
test_processes_terminate (test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest) ... 0.05s ok
test_run_after_shutdown (test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest) ... 0.00s ok
test_submit_after_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest) ... 0.42s ok
test_all_completed (test.test_concurrent_futures.ProcessPoolForkWaitTest) ... 0.16s ok
test_first_completed (test.test_concurrent_futures.ProcessPoolForkWaitTest) ... 1.70s ok
test_first_completed_some_already_completed (test.test_concurrent_futures.ProcessPoolForkWaitTest) ... 1.80s ok
test_first_exception (test.test_concurrent_futures.ProcessPoolForkWaitTest) ... 3.18s ok
test_first_exception_one_already_failed (test.test_concurrent_futures.ProcessPoolForkWaitTest) ... 2.24s ok
test_first_exception_some_already_complete (test.test_concurrent_futures.ProcessPoolForkWaitTest) ... 1.86s ok
test_timeout (test.test_concurrent_futures.ProcessPoolForkWaitTest) ... 6.26s ok
test_correct_timeout_exception_msg (test.test_concurrent_futures.ProcessPoolForkserverAsCompletedTest) ... 1.98s ok
test_duplicate_futures (test.test_concurrent_futures.ProcessPoolForkserverAsCompletedTest) ... 3.44s ok
test_free_reference_yielded_future (test.test_concurrent_futures.ProcessPoolForkserverAsCompletedTest) ... 1.76s ok
test_no_timeout (test.test_concurrent_futures.ProcessPoolForkserverAsCompletedTest) ... 1.90s ok
test_zero_timeout (test.test_concurrent_futures.ProcessPoolForkserverAsCompletedTest) ... 3.66s ok
test_crash (test.test_concurrent_futures.ProcessPoolForkserverExecutorDeadlockTest) ... 12.13s ok
test_shutdown_deadlock (test.test_concurrent_futures.ProcessPoolForkserverExecutorDeadlockTest) ... 3.03s ok
test_initializer (test.test_concurrent_futures.ProcessPoolForkserverFailingInitializerTest) ... 1.12s ok
test_initializer (test.test_concurrent_futures.ProcessPoolForkserverInitializerTest) ... 1.39s ok
test_free_reference (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 2.09s ok
test_killed_child (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 1.93s ok
test_map (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 2.02s ok
test_map_chunksize (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 1.87s ok
test_map_exception (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 1.90s ok
test_map_timeout (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 7.73s ok
test_max_workers_negative (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 1.55s ok
test_max_workers_too_large (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... skipped 'Windows-only process limit'
test_no_stale_references (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 1.78s ok
test_ressources_gced_in_workers (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 3.01s ok
test_shutdown_race_issue12456 (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 1.70s ok
test_submit (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 1.67s ok
test_submit_keyword (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 1.99s ok
test_traceback (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolExecutorTest) ... 2.03s ok
test_context_manager_shutdown (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest) ... 0.16s ok
test_del_shutdown (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest) ... 0.11s ok
test_hang_issue12364 (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest) ... 2.94s ok
test_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest) ... 3.31s ok
test_processes_terminate (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest) ... 1.53s ok
test_run_after_shutdown (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest) ... 0.00s ok
test_submit_after_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest) ... 1.50s ok
test_all_completed (test.test_concurrent_futures.ProcessPoolForkserverWaitTest) ... 2.03s ok
test_first_completed (test.test_concurrent_futures.ProcessPoolForkserverWaitTest) ... 3.41s ok
test_first_completed_some_already_completed (test.test_concurrent_futures.ProcessPoolForkserverWaitTest) ... 3.74s ok
test_first_exception (test.test_concurrent_futures.ProcessPoolForkserverWaitTest) ... 5.07s ok
test_first_exception_one_already_failed (test.test_concurrent_futures.ProcessPoolForkserverWaitTest) ... 3.91s ok
test_first_exception_some_already_complete (test.test_concurrent_futures.ProcessPoolForkserverWaitTest) ... 3.46s ok
test_timeout (test.test_concurrent_futures.ProcessPoolForkserverWaitTest) ... 7.47s ok
test_correct_timeout_exception_msg (test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTest) ... 2.44s ok
test_duplicate_futures (test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTest) ... 4.78s ok
test_free_reference_yielded_future (test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTest) ... 2.63s ok
test_no_timeout (test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTest) ... 2.74s ok
test_zero_timeout (test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTest) ... 4.35s ok
test_crash (test.test_concurrent_futures.ProcessPoolSpawnExecutorDeadlockTest) ... 17.00s ok
test_shutdown_deadlock (test.test_concurrent_futures.ProcessPoolSpawnExecutorDeadlockTest) ... 3.79s ok
test_initializer (test.test_concurrent_futures.ProcessPoolSpawnFailingInitializerTest) ... 1.78s ok
test_initializer (test.test_concurrent_futures.ProcessPoolSpawnInitializerTest) ... 1.57s ok
test_free_reference (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.47s ok
test_killed_child (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.27s ok
test_map (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.24s ok
test_map_chunksize (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.27s ok
test_map_exception (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 3.02s ok
test_map_timeout (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 8.39s ok
test_max_workers_negative (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.49s ok
test_max_workers_too_large (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... skipped 'Windows-only process limit'
test_no_stale_references (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.32s ok
test_ressources_gced_in_workers (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 4.27s ok
test_shutdown_race_issue12456 (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.21s ok
test_submit (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 3.45s ok
test_submit_keyword (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.75s ok
test_traceback (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 2.49s ok
test_context_manager_shutdown (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolShutdownTest) ... 0.07s ok
test_del_shutdown (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolShutdownTest) ... 0.14s ok
test_hang_issue12364 (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolShutdownTest) ... 3.37s ok
test_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolShutdownTest) ... 3.11s ok
test_processes_terminate (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolShutdownTest) ... 2.28s ok
test_run_after_shutdown (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolShutdownTest) ... 0.00s ok
test_submit_after_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolShutdownTest) ... 1.53s ok
test_all_completed (test.test_concurrent_futures.ProcessPoolSpawnWaitTest) ... 3.10s ok
test_first_completed (test.test_concurrent_futures.ProcessPoolSpawnWaitTest) ... 3.82s ok
test_first_completed_some_already_completed (test.test_concurrent_futures.ProcessPoolSpawnWaitTest) ... 3.92s ok
test_first_exception (test.test_concurrent_futures.ProcessPoolSpawnWaitTest) ... 4.87s ok
test_first_exception_one_already_failed (test.test_concurrent_futures.ProcessPoolSpawnWaitTest) ... 4.14s ok
test_first_exception_some_already_complete (test.test_concurrent_futures.ProcessPoolSpawnWaitTest) ... 3.48s ok
test_timeout (test.test_concurrent_futures.ProcessPoolSpawnWaitTest) ... 8.20s ok
test_correct_timeout_exception_msg (test.test_concurrent_futures.ThreadPoolAsCompletedTest) ... 0.34s ok
test_duplicate_futures (test.test_concurrent_futures.ThreadPoolAsCompletedTest) ... 2.38s ok
test_free_reference_yielded_future (test.test_concurrent_futures.ThreadPoolAsCompletedTest) ... 0.30s ok
test_no_timeout (test.test_concurrent_futures.ThreadPoolAsCompletedTest) ... 0.34s ok
test_zero_timeout (test.test_concurrent_futures.ThreadPoolAsCompletedTest) ... 2.29s ok
test_default_workers (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.45s ok
test_free_reference (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_idle_thread_reuse (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_map (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_map_exception (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_map_submits_without_iteration (test.test_concurrent_futures.ThreadPoolExecutorTest)
Tests verifying issue 11777. ... 0.11s ok
test_map_timeout (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 6.15s ok
test_max_workers_negative (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_no_stale_references (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_saturation (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.32s ok
test_shutdown_race_issue12456 (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.15s ok
test_submit (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_submit_keyword (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_initializer (test.test_concurrent_futures.ThreadPoolFailingInitializerTest) ... 0.11s ok
test_initializer (test.test_concurrent_futures.ThreadPoolInitializerTest) ... 0.19s ok
test_context_manager_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.12s ok
test_del_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.00s ok
test_hang_issue12364 (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 1.25s ok
test_interpreter_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 1.55s ok
test_run_after_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.00s ok
test_submit_after_interpreter_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.16s ok
test_thread_names_assigned (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.01s ok
test_thread_names_default (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.00s ok
test_threads_terminate (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.00s ok
test_all_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 0.11s ok
test_first_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.71s ok
test_first_completed_some_already_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.62s ok
test_first_exception (test.test_concurrent_futures.ThreadPoolWaitTests) ... 3.65s ok
test_first_exception_one_already_failed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 2.17s ok
test_first_exception_some_already_complete (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.92s ok
test_pending_calls_race (test.test_concurrent_futures.ThreadPoolWaitTests) ... 0.65s ok
test_timeout (test.test_concurrent_futures.ThreadPoolWaitTests) ... 6.69s ok

----------------------------------------------------------------------

Ran 168 tests in 303.707s

OK (skipped=3)
Warning -- files was modified by test_concurrent_futures
  Before: []
  After:  ['python.core']
msg361327 - (view) Author: Kyle Stanley (aeros) * (Python committer) Date: 2020-02-03 20:27
In GH-18228, an unrelated failure in test_concurrent_futures occurred (same as the one above):


Ran 354 tests in 496.802s
OK (skipped=35)
Warning -- files was modified by test_multiprocessing_spawn
  Before: []
  After:  ['python.core']


Full test logs: https://buildbot.python.org/all/#/builders/152/builds/259/steps/5/logs/stdio
msg363518 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-06 14:51
I identified the root cause: bpo-39877.
History
Date User Action Args
2020-03-06 14:51:46vstinnersetstatus: open -> closed
superseder: Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process
messages: + msg363518

resolution: duplicate
stage: resolved
2020-02-03 20:27:51aerossetnosy: + aeros
messages: + msg361327
2019-12-18 17:37:48vstinnercreate