classification
Title: enum with inherited type won't pickle
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ethan.furman Nosy List: Tom.Brown, ethan.furman, miss-islington
Priority: high Keywords: patch

Created on 2021-06-08 00:56 by Tom.Brown, last changed 2021-06-19 21:53 by ethan.furman. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26658 merged ethan.furman, 2021-06-10 22:19
PR 26660 merged miss-islington, 2021-06-10 22:52
PR 26666 merged ethan.furman, 2021-06-11 07:49
PR 26667 merged ethan.furman, 2021-06-11 08:05
PR 26726 merged ethan.furman, 2021-06-14 19:49
PR 26746 merged miss-islington, 2021-06-15 20:43
PR 26747 merged ethan.furman, 2021-06-15 21:23
PR 26750 merged ethan.furman, 2021-06-16 00:13
Messages (10)
msg395300 - (view) Author: Tom Brown (Tom.Brown) Date: 2021-06-08 00:56
The following script runs without error in 3.8.5 and raises an error in 3.8.6, 3.9.5 and 3.10.0b1. 

Source:
```
import enum, pickle

class MyInt(int):
    pass
    # work-around: __reduce_ex__ = int.__reduce_ex__

class MyEnum(MyInt, enum.Enum):
    A = 1

pickle.dumps(MyEnum.A)
```

Error (same in 3.8.6, 3.9.5 and 3.10.0b1):
```
Traceback (most recent call last):
  File "/home/thecap/projects/covid-data-model/./enum-pickle.py", line 12, in <module>
    pickle.dumps(MyEnum.A)
  File "/home/thecap/.pyenv/versions/3.10.0b1/lib/python3.10/enum.py", line 83, in _break_on_call_reduce
    raise TypeError('%r cannot be pickled' % self)
TypeError: MyEnum.A cannot be pickled
```

Like https://bugs.python.org/issue41889 this seems to be related to the fix for https://bugs.python.org/issue39587 which changes member_type from int to MyInt. A work-around is in the comment above.
msg395578 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-10 21:55
Looking into this I think the root of the problem is the way `reduce` is handled -- currently, Enum's `__reduce_ex__` works by returning the class, and the value to use to lookup the member.  Because that lookup can fail with complex enums, EnumType will sabotage `reduce` if it can't find support in the new enum class.

However, if `__reduce_ex__` working by returning

    `getattr, (self.__class, self._name_)`

then we should be fine, as that should never fail.
msg395585 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-10 22:52
New changeset 62f1d2b3d7dda99598d053e10b785c463fdcf591 by Ethan Furman in branch 'main':
bpo-44342: [Enum] changed pickling from by-value to by-name (GH-26658)
https://github.com/python/cpython/commit/62f1d2b3d7dda99598d053e10b785c463fdcf591
msg395589 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-10 23:37
New changeset b613132861839b6d05b67138842b579e1af29f9c by Miss Islington (bot) in branch '3.10':
bpo-44342: [Enum] changed pickling from by-value to by-name (GH-26658) (GH-26660)
https://github.com/python/cpython/commit/b613132861839b6d05b67138842b579e1af29f9c
msg395609 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-11 07:52
Changing `__reduce_ex__` is too big a change for 3.9, so making the search for pickle support more robust there.
msg395613 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-11 08:25
New changeset 3a7cccfd6cd3693e1a2ab65ee05d7f45f8501dfa by Ethan Furman in branch 'main':
bpo-44342: [Enum] fix data type search (GH-26667)
https://github.com/python/cpython/commit/3a7cccfd6cd3693e1a2ab65ee05d7f45f8501dfa
msg395615 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-11 08:30
Also fixing the search for the data type so that in

  A -> B -> C

the data type is C and not B.
msg395898 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-15 21:07
New changeset 0f99324f61d5a2edd8729be5eed6f172c892af24 by Miss Islington (bot) in branch '3.10':
bpo-44342: [Enum] fix data type search (GH-26667)
https://github.com/python/cpython/commit/0f99324f61d5a2edd8729be5eed6f172c892af24
msg395902 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-16 01:51
New changeset 41c2a4a727d3d559632598759433e0ec1bf594f5 by Ethan Furman in branch '3.10':
[3.10] bpo-44342: [Enum] improve test, add andrei kulakov to ACKS (GH-26726)
https://github.com/python/cpython/commit/41c2a4a727d3d559632598759433e0ec1bf594f5
msg395903 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-16 01:51
New changeset 741b8ae1cfc507902eb57e20f003487af13e40c0 by Ethan Furman in branch 'main':
bpo-44342: [Enum] sync current docs to 3.10 (GH-26750)
https://github.com/python/cpython/commit/741b8ae1cfc507902eb57e20f003487af13e40c0
History
Date User Action Args
2021-06-19 21:53:03ethan.furmansetstatus: open -> closed
type: behavior
resolution: fixed
stage: patch review -> resolved
2021-06-16 01:51:26ethan.furmansetmessages: + msg395903
2021-06-16 01:51:12ethan.furmansetmessages: + msg395902
2021-06-16 00:13:34ethan.furmansetpull_requests: + pull_request25335
2021-06-15 21:23:31ethan.furmansetpull_requests: + pull_request25332
2021-06-15 21:07:57ethan.furmansetmessages: + msg395898
2021-06-15 20:43:57miss-islingtonsetpull_requests: + pull_request25331
2021-06-14 19:49:30ethan.furmansetpull_requests: + pull_request25316
2021-06-11 08:30:34ethan.furmansetmessages: + msg395615
2021-06-11 08:25:30ethan.furmansetmessages: + msg395613
2021-06-11 08:05:22ethan.furmansetpull_requests: + pull_request25254
2021-06-11 07:52:25ethan.furmansetmessages: + msg395609
2021-06-11 07:49:28ethan.furmansetpull_requests: + pull_request25253
2021-06-10 23:37:37ethan.furmansetmessages: + msg395589
2021-06-10 22:52:26miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25246
2021-06-10 22:52:17ethan.furmansetmessages: + msg395585
2021-06-10 22:19:04ethan.furmansetkeywords: + patch
stage: patch review
pull_requests: + pull_request25244
2021-06-10 21:55:43ethan.furmansetmessages: + msg395578
2021-06-10 19:36:27ethan.furmansetpriority: normal -> high
versions: + Python 3.11
2021-06-08 01:12:11ethan.furmansetassignee: ethan.furman

nosy: + ethan.furman
2021-06-08 00:56:53Tom.Browncreate