➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020-11-19 13:19 by renatolfc, last changed 2022-04-11 14:59 by admin. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 23403 | merged | renatolfc, 2020-11-19 14:11 | |
| PR 23560 | merged | miss-islington, 2020-11-29 18:23 | |
| PR 23561 | merged | miss-islington, 2020-11-29 18:23 | |
| PR 28489 | closed | python-dev, 2021-09-21 11:17 | |
| Messages (7) | |||
|---|---|---|---|
| msg381410 - (view) | Author: Renato Cunha (renatolfc) * | Date: 2020-11-19 13:19 | |
Importing multiprocessing prior to other modules that define `ufunc`s breaks pickle.whichmodule for those functions.
Example:
Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing
>>> from scipy.special import gdtrix
>>> from pickle import whichmodule
>>> whichmodule(gdtrix, gdtrix.__name__)
'__mp_main__'
Now, if we change the order of imports, whichmodule works fine:
Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy.special import gdtrix
>>> from pickle import whichmodule
>>> import multiprocessing
>>> whichmodule(gdtrix, gdtrix.__name__)
'scipy.special._ufuncs'
This happens because multiprocessing creates an alias to `__main__` when imported:
Lib/multiprocessing/__init__.py
37: sys.modules['__mp_main__'] = sys.modules['__main__']
But doing so changes the ordering of `sys.modules`, and `whichmodule` returns `__mp_main__` instead of the correct `scipy.special._ufuncs`.
This bug has arisen in the context of using `dill` (which imports multiprocessing) for serialization and is further documented at https://github.com/uqfoundation/dill/issues/392.
I intend to submit a patch/PR with a fix soon.
|
|||
| msg382024 - (view) | Author: Gregory P. Smith (gregory.p.smith) *  |
Date: 2020-11-28 22:25 | |
gross / nice find :) |
|||
| msg382085 - (view) | Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2020-11-29 18:23 | |
New changeset 86684319d3dad8e1a7b0559727a48e0bc50afb01 by Renato Cunha in branch 'master': bpo-42406: Fix whichmodule() with multiprocessing (GH-23403) https://github.com/python/cpython/commit/86684319d3dad8e1a7b0559727a48e0bc50afb01 |
|||
| msg382086 - (view) | Author: miss-islington (miss-islington) | Date: 2020-11-29 18:43 | |
New changeset b1c48e513624641d6472262c33d09624170ea1f5 by Miss Islington (bot) in branch '3.8': bpo-42406: Fix whichmodule() with multiprocessing (GH-23403) https://github.com/python/cpython/commit/b1c48e513624641d6472262c33d09624170ea1f5 |
|||
| msg382088 - (view) | Author: miss-islington (miss-islington) | Date: 2020-11-29 18:47 | |
New changeset fcf7391f598e027a083173662a3ecf7865251291 by Miss Islington (bot) in branch '3.9': bpo-42406: Fix whichmodule() with multiprocessing (GH-23403) https://github.com/python/cpython/commit/fcf7391f598e027a083173662a3ecf7865251291 |
|||
| msg382097 - (view) | Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2020-11-29 23:48 | |
thanks Renato! |
|||
| msg382160 - (view) | Author: Renato Cunha (renatolfc) * | Date: 2020-11-30 16:37 | |
No worries, Gregory. Glad I could help. :-) |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:38 | admin | set | github: 86572 |
| 2021-09-21 11:17:37 | python-dev | set | nosy:
+ python-dev pull_requests: + pull_request26885 |
| 2020-11-30 16:37:02 | renatolfc | set | messages: + msg382160 |
| 2020-11-29 23:48:33 | gregory.p.smith | set | status: open -> closed resolution: fixed messages: + msg382097 stage: patch review -> commit review |
| 2020-11-29 18:47:37 | miss-islington | set | messages: + msg382088 |
| 2020-11-29 18:43:51 | miss-islington | set | messages: + msg382086 |
| 2020-11-29 18:23:41 | miss-islington | set | pull_requests: + pull_request22442 |
| 2020-11-29 18:23:33 | miss-islington | set | nosy:
+ miss-islington pull_requests: + pull_request22441 |
| 2020-11-29 18:23:25 | gregory.p.smith | set | messages: + msg382085 |
| 2020-11-28 22:25:11 | gregory.p.smith | set | assignee: gregory.p.smith messages: + msg382024 nosy: + gregory.p.smith |
| 2020-11-28 20:13:33 | epaine | set | nosy:
+ pitrou, davin versions: - Python 3.6, Python 3.7 |
| 2020-11-19 14:11:06 | renatolfc | set | keywords:
+ patch stage: patch review pull_requests: + pull_request22295 |
| 2020-11-19 14:04:57 | renatolfc | set | components: + Library (Lib), - Interpreter Core |
| 2020-11-19 13:19:31 | renatolfc | create | |