Issue42949
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2021-01-17 22:10 by macfreek, last changed 2022-04-11 14:59 by admin.
Messages (1) | |||
---|---|---|---|
msg385168 - (view) | Author: Freek Dijkstra (macfreek) | Date: 2021-01-17 22:10 | |
Summary: multiprocessing.Pool contains a bug when the script is invoked with pdb. Steps to reproduce: Consider the following script: ``` from multiprocessing import Pool def f(x): return x*x if __name__ == '__main__': with Pool(5) as p: print(p.map(f, [1, 2, 3])) ``` When called as `python3.10 test.py`, this works fine, and returns `[1, 4, 9]` When called as `python3.10 -m pdb test.py`, it gives the following exception: ``` freek@minnie ~ » python3.9 -m pdb test.py > /Users/freek/test.py(3)<module>() -> from multiprocessing import Pool (Pdb) c Traceback (most recent call last): File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pdb.py", line 1704, in main pdb._runscript(mainpyfile) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pdb.py", line 1573, in _runscript self.run(statement) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/bdb.py", line 580, in run exec(cmd, globals, locals) File "<string>", line 1, in <module> File "/Users/freek/test.py", line 3, in <module> from multiprocessing import Pool File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 119, in Pool return Pool(processes, initializer, initargs, maxtasksperchild, File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 212, in __init__ self._repopulate_pool() File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 303, in _repopulate_pool return self._repopulate_pool_static(self._ctx, self.Process, File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 326, in _repopulate_pool_static w.start() File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__ super().__init__(process_obj) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch prep_data = spawn.get_preparation_data(process_obj._name) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 183, in get_preparation_data main_mod_name = getattr(main_module.__spec__, "name", None) AttributeError: module '__main__' has no attribute '__spec__' ``` I can reproduce this in Python 3.8.7, 3.9.1, 3.10.0a3. It works fine in Python 3.7.9. A workaround is to define `__spec__` in __main__: ``` from multiprocessing import Pool def f(x): return x*x if __name__ == '__main__': __spec__ = None with Pool(5) as p: print(p.map(f, [1, 2, 3])) ``` I'd say that multiprocessing/spawn.py", line 183, in get_preparation_data is flawed main_mod_name = getattr(main_module.__spec__, "name", None) |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:40 | admin | set | github: 87115 |
2021-01-17 22:10:12 | macfreek | create |