This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Unable to pickle enum with nested frozen dataclass?
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Eric Cousineau, eric.smith, ethan.furman
Priority: normal Keywords:

Created on 2021-10-20 23:12 by Eric Cousineau, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
enum_test.py Eric Cousineau, 2021-10-20 23:12 Example showing positives vs. failing for class / enum pickling
enum_nested_type_test.py Eric Cousineau, 2021-10-21 18:32
Messages (6)
msg404539 - (view) Author: Eric Cousineau (Eric Cousineau) * Date: 2021-10-20 23:12
I seem cannot pickle enum values from an enum class w/ a nested frozen dataclass.

I can pickle a field from a normal class w/ a nested frozen dataclass, and I can pickle a field from an enum with a top-level frozen dataclass - perhaps it's some interplay with `__qualname__` and `enum.py`?.

See attached file for sample test.
I checked on 3.6.9, 3.9.7, and 3.10.0.

Looked around, couldn't easily find existing issue for this.
msg404541 - (view) Author: Eric Cousineau (Eric Cousineau) * Date: 2021-10-20 23:15
Ah, forgot to include error message:

```
<class '__main__.Works1'>
<enum 'Works2'>
<enum 'DoesNotWork'>
E
...
_pickle.PicklingError: Can't pickle <class '__main__.DoesNotWork.NestedValue2'>: it's not the same object as __main__.DoesNotWork.NestedValue2
```
msg404543 - (view) Author: Eric Cousineau (Eric Cousineau) * Date: 2021-10-20 23:27
Thinking on it some more, I the current `Enum` metaclass doesn't distinguish between nested class and normal field, so it wraps it, thus why `pickle` gets confused.

Perhaps it's possible to recognize this case - the class w/ `__qualname__` can be checked to see if it's nested.

If so, then options are (a) skip wrapping the field, (b) failing fast, or (c) "no fix"?
msg404558 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-10-21 00:31
I get an error with 3.8.10, but not on the main branch (3.11). I don't have other versions handy to test.

Removing 3.6, as it's no longer getting bug fixes, and this doesn't look like a security issue.
msg404562 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-10-21 02:22
I'm confused -- your initial report talks about pickling fields, but your tests are pickling the entire class... what am I missing?
msg404632 - (view) Author: Eric Cousineau (Eric Cousineau) * Date: 2021-10-21 18:32
> I get an error with 3.8.10, but not on the main branch (3.11) [...]

Confirmed, using 3.11.0a1 (using Docker). Seems like at least the pickling is fixed?

> I'm confused -- your initial report talks about pickling fields, but your tests are pickling the entire class... what am I missing?

Sorry about that; the original file, `enum_test.py`, was just trying to pickle an enum field, i.e. `DoesNotWork.a`. However, because it uses the type `DoesNotWork.NestedValue2`, `pickle` tries to unpackage that type, but cannot access it via name because `DoesNotWork.NestedValue2` is wrapped as an enum *field*, rather than just a normal field.

I've uploaded a new file, `enum_nested_type_test.py`, that perhaps shows the root cause - the `Enum` interprets the nested class as an enum field (wrapping it as an enum value), rather than as "just" a nested class. In contrast, it did not seem to wrap the function.

Does that make sense / help at all?
History
Date User Action Args
2022-04-11 14:59:51adminsetgithub: 89709
2021-11-04 04:37:08ethan.furmansetmessages: - msg405650
2021-11-03 21:04:21ethan.furmansetmessages: + msg405650
2021-10-21 18:32:21Eric Cousineausetfiles: + enum_nested_type_test.py

messages: + msg404632
2021-10-21 02:22:18ethan.furmansetmessages: + msg404562
2021-10-21 00:45:08eric.smithsetnosy: + ethan.furman
2021-10-21 00:31:31eric.smithsetnosy: + eric.smith

messages: + msg404558
versions: - Python 3.6
2021-10-20 23:27:43Eric Cousineausetmessages: + msg404543
2021-10-20 23:15:55Eric Cousineausetmessages: + msg404541
2021-10-20 23:12:50Eric Cousineaucreate