classification
Title: enum with inherited type won't pickle
Type: Stage: patch review
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: ethan.furman Nosy List: Tom.Brown, ethan.furman, miss-islington
Priority: high Keywords: patch

Created on 2021-06-08 00:56 by Tom.Brown, last changed 2021-06-11 08:30 by ethan.furman.

Pull Requests
URL Status Linked Edit
PR 26658 merged ethan.furman, 2021-06-10 22:19
PR 26660 merged miss-islington, 2021-06-10 22:52
PR 26666 merged ethan.furman, 2021-06-11 07:49
PR 26667 merged ethan.furman, 2021-06-11 08:05
Messages (7)
msg395300 - (view) Author: Tom Brown (Tom.Brown) Date: 2021-06-08 00:56
The following script runs without error in 3.8.5 and raises an error in 3.8.6, 3.9.5 and 3.10.0b1. 

Source:
```
import enum, pickle

class MyInt(int):
    pass
    # work-around: __reduce_ex__ = int.__reduce_ex__

class MyEnum(MyInt, enum.Enum):
    A = 1

pickle.dumps(MyEnum.A)
```

Error (same in 3.8.6, 3.9.5 and 3.10.0b1):
```
Traceback (most recent call last):
  File "/home/thecap/projects/covid-data-model/./enum-pickle.py", line 12, in <module>
    pickle.dumps(MyEnum.A)
  File "/home/thecap/.pyenv/versions/3.10.0b1/lib/python3.10/enum.py", line 83, in _break_on_call_reduce
    raise TypeError('%r cannot be pickled' % self)
TypeError: MyEnum.A cannot be pickled
```

Like https://bugs.python.org/issue41889 this seems to be related to the fix for https://bugs.python.org/issue39587 which changes member_type from int to MyInt. A work-around is in the comment above.
msg395578 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-10 21:55
Looking into this I think the root of the problem is the way `reduce` is handled -- currently, Enum's `__reduce_ex__` works by returning the class, and the value to use to lookup the member.  Because that lookup can fail with complex enums, EnumType will sabotage `reduce` if it can't find support in the new enum class.

However, if `__reduce_ex__` working by returning

    `getattr, (self.__class, self._name_)`

then we should be fine, as that should never fail.
msg395585 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-10 22:52
New changeset 62f1d2b3d7dda99598d053e10b785c463fdcf591 by Ethan Furman in branch 'main':
bpo-44342: [Enum] changed pickling from by-value to by-name (GH-26658)
https://github.com/python/cpython/commit/62f1d2b3d7dda99598d053e10b785c463fdcf591
msg395589 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-10 23:37
New changeset b613132861839b6d05b67138842b579e1af29f9c by Miss Islington (bot) in branch '3.10':
bpo-44342: [Enum] changed pickling from by-value to by-name (GH-26658) (GH-26660)
https://github.com/python/cpython/commit/b613132861839b6d05b67138842b579e1af29f9c
msg395609 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-11 07:52
Changing `__reduce_ex__` is too big a change for 3.9, so making the search for pickle support more robust there.
msg395613 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-11 08:25
New changeset 3a7cccfd6cd3693e1a2ab65ee05d7f45f8501dfa by Ethan Furman in branch 'main':
bpo-44342: [Enum] fix data type search (GH-26667)
https://github.com/python/cpython/commit/3a7cccfd6cd3693e1a2ab65ee05d7f45f8501dfa
msg395615 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-06-11 08:30
Also fixing the search for the data type so that in

  A -> B -> C

the data type is C and not B.
History
Date User Action Args
2021-06-11 08:30:34ethan.furmansetmessages: + msg395615
2021-06-11 08:25:30ethan.furmansetmessages: + msg395613
2021-06-11 08:05:22ethan.furmansetpull_requests: + pull_request25254
2021-06-11 07:52:25ethan.furmansetmessages: + msg395609
2021-06-11 07:49:28ethan.furmansetpull_requests: + pull_request25253
2021-06-10 23:37:37ethan.furmansetmessages: + msg395589
2021-06-10 22:52:26miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25246
2021-06-10 22:52:17ethan.furmansetmessages: + msg395585
2021-06-10 22:19:04ethan.furmansetkeywords: + patch
stage: patch review
pull_requests: + pull_request25244
2021-06-10 21:55:43ethan.furmansetmessages: + msg395578
2021-06-10 19:36:27ethan.furmansetpriority: normal -> high
versions: + Python 3.11
2021-06-08 01:12:11ethan.furmansetassignee: ethan.furman

nosy: + ethan.furman
2021-06-08 00:56:53Tom.Browncreate