classification
Title: Weird exception behaviour in ProcessPoolExecutor
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: duplicate
Dependencies: Superseder: picke cannot dump Exception subclasses with different super() args
View: 37287
Assigned To: bquinlan Nosy List: Iceflower, bquinlan, pitrou, raabf
Priority: normal Keywords:

Created on 2019-06-09 11:04 by Iceflower, last changed 2019-06-28 18:30 by bquinlan. This issue is now closed.

Files
File name Uploaded Description Edit
ProcPoolEx-exception.py Iceflower, 2019-06-09 11:04 Example code
Messages (2)
msg345079 - (view) Author: Iceflower (Iceflower) Date: 2019-06-09 11:04
I don't really know where this belongs, but it is at least for me not an expected behaviour. It is weird for me at all. Please take a look if this is an intended behaviour and why it is like that.

Tested with py3.7.3, py3.8.0b1
```
A process in the process pool was terminated abruptly while the future was running or pending.
```

Tested with py 3.6.8:
```
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Python36\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Python36\lib\threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python36\lib\concurrent\futures\process.py", line 272, in _queue_management_worker
    result_item = reader.recv()
  File "C:\Python36\lib\multiprocessing\connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
TypeError: __init__() missing 1 required positional argument: 'num'
```

I expect that the PoolBreaker exception would work too.

Code to test:
As attachment.
msg345642 - (view) Author: Brian Quinlan (bquinlan) * (Python committer) Date: 2019-06-14 22:05
That's a super interesting bug! It looks like this issue is that your exception can't be round-tripped using pickle i.e.

>>> class PoolBreaker(Exception):
...     def __init__(self, num):
...         super().__init__()
...         self.num = num
... 
>>> import pickle
>>> pickle.loads(pickle.dumps(PoolBreaker(5)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'num'
History
Date User Action Args
2019-06-28 18:30:41bquinlansetstatus: open -> closed
stage: resolved
2019-06-28 16:26:27raabfsetnosy: + raabf
2019-06-14 23:33:41bquinlansetsuperseder: picke cannot dump Exception subclasses with different super() args
resolution: duplicate
2019-06-14 22:05:05bquinlansetassignee: bquinlan
messages: + msg345642
2019-06-09 18:31:23xtreaksetnosy: + bquinlan, pitrou
2019-06-09 11:04:00Iceflowercreate