Created on 2014-07-02 13:03 by Olivier.Grisel, last changed 2014-10-06 07:25 by Olivier.Grisel. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| pickle_whichmodule.patch | Olivier.Grisel, 2014-07-02 13:03 | review | ||
| pickle_whichmodule_20140703.patch | Olivier.Grisel, 2014-07-03 15:04 | review | ||
| Messages (6) | |||
|---|---|---|---|
| msg222099 - (view) | Author: Olivier Grisel (Olivier.Grisel) * | Date: 2014-07-02 13:03 | |
`pickle.whichmodule` performs an iteration over `sys.modules` and tries to perform `getattr` calls on those modules. Unfortunately some modules such as those from the `six.moves` dynamic module can trigger imports when calling `getattr` on them, hence mutating the `sys.modules` dict and causing a `RuntimeError: dictionary changed size during iteration`. This would also render `pickle.whichmodule` more thread-safe and cause concurrent thread perform new module imports and `whichmodule` calls. The attach patch protect the iterator by copying the dict items into a fixed list. I could write a tests involving dynamic module definitions as done in `six.moves` but it sounds very complicated for such a trivial fix. |
|||
| msg222188 - (view) | Author: Antoine Pitrou (pitrou) *  |
Date: 2014-07-03 14:39 | |
Sounds fine, but you should at least add a comment explaining the list() call. |
|||
| msg222193 - (view) | Author: Olivier Grisel (Olivier.Grisel) * | Date: 2014-07-03 15:04 | |
New version of the patch to add an inline comment. |
|||
| msg228488 - (view) | Author: Roundup Robot (python-dev) | Date: 2014-10-04 20:17 | |
New changeset 86ba3bdfac15 by Antoine Pitrou in branch '3.4': Issue #21905: Avoid RuntimeError in pickle.whichmodule() when sys.modules is mutated while iterating. https://hg.python.org/cpython/rev/86ba3bdfac15 New changeset d748a3503ad5 by Antoine Pitrou in branch 'default': Issue #21905: Avoid RuntimeError in pickle.whichmodule() when sys.modules is mutated while iterating. https://hg.python.org/cpython/rev/d748a3503ad5 |
|||
| msg228489 - (view) | Author: Antoine Pitrou (pitrou) * |
Date: 2014-10-04 20:18 | |
Sorry for the delay, Olivier. Your patch is now pushed. Thank you! |
|||
| msg228646 - (view) | Author: Olivier Grisel (Olivier.Grisel) * | Date: 2014-10-06 07:25 | |
No problem. Thanks Antoine for the review! |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2014-10-06 15:10:31 | haypo | link | issue22292 superseder |
| 2014-10-06 07:25:56 | Olivier.Grisel | set | messages: + msg228646 |
| 2014-10-04 20:18:24 | pitrou | set | status: open -> closed resolution: fixed messages: + msg228489 stage: patch review -> resolved |
| 2014-10-04 20:17:51 | python-dev | set | nosy:
+ python-dev messages: + msg228488 |
| 2014-07-03 15:04:30 | Olivier.Grisel | set | files:
+ pickle_whichmodule_20140703.patch messages: + msg222193 |
| 2014-07-03 14:39:30 | pitrou | set | nosy:
+ pitrou, alexandre.vassalotti messages: + msg222188 type: crash -> behavior stage: patch review |
| 2014-07-02 13:03:12 | Olivier.Grisel | create | |