Issue43768
Created on 2021-04-07 19:12 by jerryc05, last changed 2021-04-07 19:12 by jerryc05.
Messages (1) | |||
---|---|---|---|
msg390468 - (view) | Author: jerryc05 (jerryc05) | Date: 2021-04-07 19:12 | |
I am not sure if this is a bug or an expected case. Long story short, I tried to print the content of a `multiprocessing.Manager().dict()` in the main thread, but I got a strange error. I encountered this error only when the number of pools is rather large (>20) and only on `macOS` (works perfectly on `Linux`). Specs: ``` CPU: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz macOS: 11.2.3 ``` The minimum err code is attached: ```python3 #!/usr/bin/env python3 from contextlib import suppress import multiprocessing as mp import time def run(): D[mp.current_process().name] = 'some val' time.sleep(0.5) if __name__ == '__main__': mp.set_start_method('fork') D, rets = mp.Manager().dict(), [] with mp.Pool(25) as p: for _ in range(33): rets.append(p.apply_async(run)) while rets: for ret in rets[:]: with suppress(mp.TimeoutError): ret.get(timeout=0) rets.remove(ret) print(len(D)) ``` Error: ``` multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 801, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/Users/???", line 9, in run D[mp.current_process().name] = 'some val' File "<string>", line 2, in __setitem__ File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 805, in _callmethod self._connect() File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 792, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 507, in Client c = SocketClient(address) File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 61] Connection refused """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/???", line 22, in <module> ret.get(timeout=0) File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 771, in get raise self._value ConnectionRefusedError: [Errno 61] Connection refused ``` |
History | |||
---|---|---|---|
Date | User | Action | Args |
2021-04-07 19:12:33 | jerryc05 | create |