This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ProcessPoolExecutor().map() fails following an identical map()
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: OceanEngineer, ned.deily
Priority: normal Keywords:

Created on 2015-05-16 22:53 by OceanEngineer, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
processExample.py OceanEngineer, 2015-05-16 22:53 bug creation
Messages (2)
msg243375 - (view) Author: OceanEngineer (OceanEngineer) Date: 2015-05-16 22:53
Attached file runs fine. Uncommenting line 19 makes python crash. Also, no crash happens if order of execution of map() and executor.map() is switched.

This problem seems to be related to numpy in some way, the code does not crash if the commented return of function user_square is substituted for numpy functions.

Using a mac with Yosemite 10.10.3. When program crashed I get window that says:

"
Python quit unexpectedly.

Click Reopen to open the application again. Click Report to see more detailed information and send a report to Apple.
"

This is the traceback:

Traceback (most recent call last):
  File "processExample.py", line 27, in <module>
    main()
  File "processExample.py", line 22, in main
    result3 = list(executor.map(user_square, nums))
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 549, in result_iterator
    yield future.result()
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 402, in result
    return self.__get_result()
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 354, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
msg243383 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-05-17 04:37
I can reproduce a crash with your sample program using the current numpy wheel (1.9.2) and the python.org 3.4.3.  The significant parts of the OS X crash dump:

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000110

VM Regions Near 0x110:
--> 
    __TEXT                 0000000100000000-0000000100001000 [    4K] r-x/rwx SM=COW  /Library/Frameworks/Python.framework/Versions/3.4.3_10_6/Resources/Python.app/Contents/MacOS/Python

Application Specific Information:
crashed on child side of fork pre-exec

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libdispatch.dylib             	0x00007fff97347c13 dispatch_group_async + 533
1   libBLAS.dylib                 	0x00007fff96271a67 dsyrk_r + 565
2   libBLAS.dylib                 	0x00007fff96272183 APL_dsyrk + 370
3   libBLAS.dylib                 	0x00007fff96370696 cblas_dgemm + 1144
4   _dotblas.so                   	0x0000000104216775 dotblas_matrixproduct + 5733
5   org.python.python             	0x00000001000e44ce PyEval_EvalFrameEx + 28894
6   org.python.python             	0x00000001000e5ced PyEval_EvalCodeEx + 2349
7   org.python.python             	0x000000010003e8ba function_call + 186
8   org.python.python             	0x000000010000d3c8 PyObject_Call + 104
9   org.python.python             	0x00000001000e0cb9 PyEval_EvalFrameEx + 14537
10  org.python.python             	0x00000001000e5ced PyEval_EvalCodeEx + 2349
11  org.python.python             	0x000000010003e8ba function_call + 186
12  org.python.python             	0x000000010000d3c8 PyObject_Call + 104
13  org.python.python             	0x00000001000e0cb9 PyEval_EvalFrameEx + 14537
14  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
15  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
16  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
17  org.python.python             	0x00000001000e5ced PyEval_EvalCodeEx + 2349
18  org.python.python             	0x000000010003e8ba function_call + 186
19  org.python.python             	0x000000010000d3c8 PyObject_Call + 104
20  org.python.python             	0x000000010002802c method_call + 140
21  org.python.python             	0x000000010000d3c8 PyObject_Call + 104
22  org.python.python             	0x000000010007b831 slot_tp_init + 81
23  org.python.python             	0x0000000100072d14 type_call + 212
24  org.python.python             	0x000000010000d3c8 PyObject_Call + 104
25  org.python.python             	0x00000001000e1b09 PyEval_EvalFrameEx + 18201
26  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
27  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
28  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
29  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
30  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
31  org.python.python             	0x00000001000e5ced PyEval_EvalCodeEx + 2349
32  org.python.python             	0x000000010003e8ba function_call + 186
33  org.python.python             	0x000000010000d3c8 PyObject_Call + 104
34  org.python.python             	0x00000001000e0cb9 PyEval_EvalFrameEx + 14537
35  org.python.python             	0x00000001000e5ced PyEval_EvalCodeEx + 2349
36  org.python.python             	0x00000001000e4306 PyEval_EvalFrameEx + 28438
37  org.python.python             	0x00000001000e5ced PyEval_EvalCodeEx + 2349
38  org.python.python             	0x00000001000e4306 PyEval_EvalFrameEx + 28438
39  org.python.python             	0x00000001000e4455 PyEval_EvalFrameEx + 28773
40  org.python.python             	0x00000001000e5ced PyEval_EvalCodeEx + 2349
41  org.python.python             	0x00000001000e5daf PyEval_EvalCode + 63
42  org.python.python             	0x000000010011048e PyRun_FileExFlags + 206
43  org.python.python             	0x000000010011083d PyRun_SimpleFileExFlags + 717
44  org.python.python             	0x000000010012810e Py_Main + 3262
45  org.python.python             	0x0000000100000e32 0x100000000 + 3634
46  org.python.python             	0x0000000100000c84 0x100000000 + 3204

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000105001480  rbx: 0x0000000000000000  rcx: 0x0000000000000000  rdx: 0x0000000000000100
  rdi: 0x00007fff7cd5fd40  rsi: 0x0000000000000000  rbp: 0x00007fff5bffbbd0  rsp: 0x00007fff5bffbba0
   r8: 0x0000000105001400   r9: 0x0000000000000000  r10: 0x00000000000007d0  r11: 0xffff8001a446c4d8
  r12: 0x00000001004682b0  r13: 0x0000000100437210  r14: 0x00007fff97341317  r15: 0x0000000000002000
  rip: 0x00007fff97347c13  rfl: 0x0000000000010206  cr2: 0x0000000000000110
  
Logical CPU:     2
Error Code:      0x00000006
Trap Number:     14
[...]
       0x104212000 -        0x104217ff7 +_dotblas.so (???) <15CB5D81-FD91-3005-A21C-A7D1432020C2> /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/numpy/core/_dotblas.so
[...]
    0x7fff96258000 -     0x7fff963e6fff  libBLAS.dylib (1128) <497912C1-A98E-3281-BED7-E9C751552F61> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
[...]

So the crash is occurring as a result of a call from NumPy's _dotblas extension module to OS X's libBLAS.  There's nothing obvious here that would indicate this is a problem in Python itself.  Suggest you pursue this in the NumPy community and, if necessary, open an issue there:

http://www.scipy.org/scipylib/mailing-lists.html
History
Date User Action Args
2022-04-11 14:58:16adminsetgithub: 68401
2015-05-17 04:37:18ned.deilysetstatus: open -> closed

nosy: + ned.deily
messages: + msg243383

resolution: third party
stage: resolved
2015-05-16 22:53:14OceanEngineercreate