Message381410
Importing multiprocessing prior to other modules that define `ufunc`s breaks pickle.whichmodule for those functions.
Example:
Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing
>>> from scipy.special import gdtrix
>>> from pickle import whichmodule
>>> whichmodule(gdtrix, gdtrix.__name__)
'__mp_main__'
Now, if we change the order of imports, whichmodule works fine:
Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy.special import gdtrix
>>> from pickle import whichmodule
>>> import multiprocessing
>>> whichmodule(gdtrix, gdtrix.__name__)
'scipy.special._ufuncs'
This happens because multiprocessing creates an alias to `__main__` when imported:
Lib/multiprocessing/__init__.py
37: sys.modules['__mp_main__'] = sys.modules['__main__']
But doing so changes the ordering of `sys.modules`, and `whichmodule` returns `__mp_main__` instead of the correct `scipy.special._ufuncs`.
This bug has arisen in the context of using `dill` (which imports multiprocessing) for serialization and is further documented at https://github.com/uqfoundation/dill/issues/392.
I intend to submit a patch/PR with a fix soon. |
|
Date |
User |
Action |
Args |
2020-11-19 13:19:31 | renatolfc | set | recipients:
+ renatolfc |
2020-11-19 13:19:31 | renatolfc | set | messageid: <1605791971.73.0.31216879751.issue42406@roundup.psfhosted.org> |
2020-11-19 13:19:31 | renatolfc | link | issue42406 messages |
2020-11-19 13:19:31 | renatolfc | create | |
|