This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_concurrent_futures crashes on 32-bit Windows Vista
Type: crash Stage: resolved
Components: Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, tim.peters, zach.ware
Priority: normal Keywords:

Created on 2013-11-25 21:36 by tim.peters, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (8)
msg204418 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-25 21:36
Worked OK yesterday, using current default branch in all cases.

C:\Code\Python\PCbuild>.\\python_d  -Wd -E -bb ../lib/test/regrtest.py test_concurrent_futures
[1/1] test_concurrent_futures
Fatal Python error: Segmentation fault

Current thread 0x00000590 (most recent call first):
  File "<frozen importlib._bootstrap>", line 882 in spec_from_file_location
  File "<frozen importlib._bootstrap>", line 1930 in _get_spec
  File "<frozen importlib._bootstrap>", line 1971 in find_spec
  File "<frozen importlib._bootstrap>", line 1834 in _get_spec
  File "<frozen importlib._bootstrap>", line 1860 in find_spec
  File "<frozen importlib._bootstrap>", line 2074 in _find_spec
  File "<frozen importlib._bootstrap>", line 2132 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 2147 in _find_and_load
  File "<frozen importlib._bootstrap>", line 303 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 2194 in _handle_fromlist
  File "C:\Code\Python\lib\multiprocessing\popen_spawn_win32.py", line 8 in <module>
  File "<frozen importlib._bootstrap>", line 303 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1422 in exec_module
  File "<frozen importlib._bootstrap>", line 1111 in _exec
  File "<frozen importlib._bootstrap>", line 1187 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 2136 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 2147 in _find_and_load
  File "C:\Code\Python\lib\multiprocessing\context.py", line 312 in _Popen
  File "C:\Code\Python\lib\multiprocessing\context.py", line 212 in _Popen
  File "C:\Code\Python\lib\multiprocessing\process.py", line 105 in start
  File "C:\Code\Python\lib\concurrent\futures\process.py", line 387 in _adjust_process_count
  File "C:\Code\Python\lib\concurrent\futures\process.py", line 368 in _start_queue_management_thread
  File "C:\Code\Python\lib\concurrent\futures\process.py", line 407 in submit
  File "C:\Code\Python\lib\test\test_concurrent_futures.py", line 83 in <listcomp>
  File "C:\Code\Python\lib\test\test_concurrent_futures.py", line 83 in _prime_executor
  File "C:\Code\Python\lib\test\test_concurrent_futures.py", line 70 in setUp
  File "C:\Code\Python\lib\unittest\case.py", line 567 in run
  File "C:\Code\Python\lib\unittest\case.py", line 610 in __call__
  File "C:\Code\Python\lib\unittest\suite.py", line 117 in run
  File "C:\Code\Python\lib\unittest\suite.py", line 79 in __call__
  File "C:\Code\Python\lib\unittest\suite.py", line 117 in run
  File "C:\Code\Python\lib\unittest\suite.py", line 79 in __call__
  File "C:\Code\Python\lib\unittest\suite.py", line 117 in run
  File "C:\Code\Python\lib\unittest\suite.py", line 79 in __call__
  File "C:\Code\Python\lib\test\support\__init__.py", line 1584 in run
  File "C:\Code\Python\lib\test\support\__init__.py", line 1685 in _run_suite
  File "C:\Code\Python\lib\test\support\__init__.py", line 1719 in run_unittest
  File "C:\Code\Python\lib\test\test_concurrent_futures.py", line 673 in test_main
  File "C:\Code\Python\lib\test\support\__init__.py", line 1831 in decorator
  File "../lib/test/regrtest.py", line 1278 in runtest_inner
  File "../lib/test/regrtest.py", line 978 in runtest
  File "../lib/test/regrtest.py", line 763 in main
  File "../lib/test/regrtest.py", line 1562 in main_in_temp_cwd
  File "../lib/test/regrtest.py", line 1587 in <module>

Running with -v suggests it's dying in test_submit_keyword:

...
test_no_stale_references (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.88s ok
test_shutdown_race_issue12456 (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.88s ok
test_submit (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.88s ok
test_submit_keyword (test.test_concurrent_futures.ProcessPoolExecutorTest) ...

And then a box pops up complaining that python_d stopped working.
msg204420 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-25 21:43
Hmm.  Looks like it's dying in gcmodule.c's visit_decref(), here:

    if (PyObject_IS_GC(op)) {

So it may or may not trigger depending on the vagaries of when cyclic gc runs.  For op, op->_ob_next, _ob_prev, ob_refcnt, and ob_type are all 0xdddddddd.  That doesn't look good - LOL ;-)

Here's the call stack:

>	python34_d.dll!visit_decref(_object * op, void * data)  Line 373 + 0x6 bytes	C
 	python34_d.dll!tupletraverse(PyTupleObject * o, int (_object *, void *)* visit, void * arg)  Line 564 + 0x1f bytes	C
 	python34_d.dll!subtract_refs(_gc_head * containers)  Line 400 + 0x11 bytes	C
 	python34_d.dll!collect(int generation, int * n_collected, int * n_uncollectable, int nofail)  Line 957 + 0x9 bytes	C
 	python34_d.dll!collect_with_callback(int generation)  Line 1128 + 0x13 bytes	C
 	python34_d.dll!collect_generations()  Line 1151 + 0x9 bytes	C
 	python34_d.dll!_PyObject_GC_Malloc(unsigned int basicsize)  Line 1740	C
 	python34_d.dll!_PyObject_GC_New(_typeobject * tp)  Line 1749 + 0xc bytes	C
 	python34_d.dll!mbuf_alloc()  Line 68 + 0xa bytes	C
 	python34_d.dll!_PyManagedBuffer_FromObject(_object * base)  Line 84 + 0x5 bytes	C
 	python34_d.dll!PyMemoryView_FromObject(_object * v)  Line 789 + 0x9 bytes	C
 	python34_d.dll!bytesio_getbuffer(bytesio * self)  Line 220 + 0x9 bytes	C
 	python34_d.dll!call_function(_object * * * pp_stack, int oparg)  Line 4208 + 0x16 bytes	C
 	python34_d.dll!PyEval_EvalFrameEx(_frame * f, int throwflag)  Line 2828	C
 	python34_d.dll!PyEval_EvalCodeEx(_object * _co, _object * globals, _object * locals, _object * * args, int argcount, _object * * kws, int kwcount, _object * * defs, int defcount, _object * kwdefs, _object * closure)  Line 3576 + 0xb bytes	C
 	python34_d.dll!fast_function(_object * func, _object * * * pp_stack, int n, int na, int nk)  Line 4335 + 0x38 bytes	C
 	python34_d.dll!call_function(_object * * * pp_stack, int oparg)  Line 4250 + 0x15 bytes	C
 	python34_d.dll!PyEval_EvalFrameEx(_frame * f, int throwflag)  Line 2828	C
 	python34_d.dll!PyEval_EvalCodeEx(_object * _co, _object * globals, _object * locals, _object * * args, int argcount, _object * * kws, int kwcount, _object * * defs, int defcount, _object * kwdefs, _object * closure)  Line 3576 + 0xb bytes	C
 	python34_d.dll!function_call(_object * func, _object * arg, _object * kw)  Line 638 + 0x41 bytes	C
 	python34_d.dll!PyObject_Call(_object * func, _object * arg, _object * kw)  Line 2087 + 0xf bytes	C
 	python34_d.dll!ext_do_call(_object * func, _object * * * pp_stack, int flags, int na, int nk)  Line 4549 + 0x8 bytes	C
 	python34_d.dll!PyEval_EvalFrameEx(_frame * f, int throwflag)  Line 2869	C
 	python34_d.dll!fast_function(_object * func, _object * * * pp_stack, int n, int na, int nk)  Line 4323	C
 	python34_d.dll!call_function(_object * * * pp_stack, int oparg)  Line 4250 + 0x15 bytes	C
 	python34_d.dll!PyEval_EvalFrameEx(_frame * f, int throwflag)  Line 2828	C
 	python34_d.dll!fast_function(_object * func, _object * * * pp_stack, int n, int na, int nk)  Line 4323	C
 	python34_d.dll!call_function(_object * * * pp_stack, int oparg)  Line 4250 + 0x15 bytes	C
 	python34_d.dll!PyEval_EvalFrameEx(_frame * f, int throwflag)  Line 2828	C
 	python34_d.dll!PyEval_EvalCodeEx(_object * _co, _object * globals, _object * locals, _object * * args, int argcount, _object * * kws, int kwcount, _object * * defs, int defcount, _object * kwdefs, _object * closure)  Line 3576 + 0xb bytes	C
 	python34_d.dll!function_call(_object * func, _object * arg, _object * kw)  Line 638 + 0x41 bytes	C
 	python34_d.dll!PyObject_Call(_object * func, _object * arg, _object * kw)  Line 2087 + 0xf bytes	C
 	python34_d.dll!method_call(_object * func, _object * arg, _object * kw)  Line 347 + 0x11 bytes	C
 	python34_d.dll!PyObject_Call(_object * func, _object * arg, _object * kw)  Line 2087 + 0xf bytes	C
 	python34_d.dll!PyEval_CallObjectWithKeywords(_object * func, _object * arg, _object * kw)  Line 4102	C
 	python34_d.dll!t_bootstrap(void * boot_raw)  Line 1000 + 0x1a bytes	C
 	python34_d.dll!bootstrap(void * call)  Line 175 + 0x7 bytes	C
 	msvcr100d.dll!_callthreadstartex()  Line 314 + 0xf bytes	C
 	msvcr100d.dll!_threadstartex(void * ptd)  Line 297	C
 	kernel32.dll!76fdd2e9() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]	
 	ntdll.dll!77181603() 	
 	ntdll.dll!771815d6()
msg204421 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-11-25 21:44
I suspect you pulled at just the wrong time and caught e39db21df580, which Alexandre has already reverted in 2a1de922651a.
msg204422 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-25 21:49
BTW, I believe 0xdddddddd is what MS's debug libraries use to overwrite memory that's already been free()'d - akin to PyMalloc's "DEADBYTE".
msg204423 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-25 21:52
Zach, after pulling again and rebuilding, still failing in what looks like the same way.
msg204425 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-11-25 21:56
Hmmm, that's odd; I can't reproduce on Windows 7 from 2a1de922651a, but I can from e39db21df580.
msg204427 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-25 22:02
Zach, that could be - my box is *really* screwed up now - going to reboot and try again.
msg204429 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-25 22:21
Yup, a reboot & rebuild fixed it.  I think stray processes left over from older test runs were preventing a rebuild from replacing my debug DLL.  Gotta love Windows ;-)
History
Date User Action Args
2022-04-11 14:57:54adminsetgithub: 63978
2013-11-25 22:21:52tim.peterssetstatus: open -> closed
type: crash
messages: + msg204429

resolution: fixed
stage: resolved
2013-11-25 22:02:19tim.peterssetmessages: + msg204427
2013-11-25 21:56:31zach.waresetmessages: + msg204425
2013-11-25 21:52:35tim.peterssetmessages: + msg204423
2013-11-25 21:49:34tim.peterssetmessages: + msg204422
2013-11-25 21:45:23zach.waresetnosy: + alexandre.vassalotti
2013-11-25 21:44:26zach.waresetnosy: + zach.ware
messages: + msg204421
2013-11-25 21:43:12tim.peterssetmessages: + msg204420
2013-11-25 21:36:19tim.peterscreate