classification
Title: enum.Enum reference count leaks
Type: resource usage Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: ethan.furman Nosy List: barry, dan.gass@gmail.com, eli.bendersky, ethan.furman, hongweipeng
Priority: low Keywords:

Created on 2020-01-25 06:30 by dan.gass@gmail.com, last changed 2020-02-10 04:26 by hongweipeng.

Files
File name Uploaded Description Edit
sample.py dan.gass@gmail.com, 2020-01-25 06:30 demonstrate refcount leak and likely root cause pattern
Messages (2)
msg360670 - (view) Author: Dan Gass (dan.gass@gmail.com) Date: 2020-01-25 06:30
Given
(1) instantiation of an enumeration class with an invalid value
(2) a try/except around the instantiation where the exception is ignored

Then:
An unneeded reference to the bad value is lost (as well as other values that I suspect are local variables within a participating method)

When run, the attached sample script shows before and after reference counts which demonstrate the potential resource leaks. The sample script includes the output from running the script on Python version 3.7.5 within the module docstring.

The root cause appears to be in the exception handling in the Enum.__new__ method (in the area where it calls the _missing_ hook). The attached sample script includes a simplified version of those methods that should help pinpoint the code in question and confirm the root cause.

Not being an exception nitty-gritty expert, I have suspicions that users should be warned about using this pattern of exception handling. I suspect this pattern would be worth avoiding in the Enum implementation.

I am willing to take a stab at submitting a patch for Enum. I hesitate slightly not knowing if there are specific reasons for the code existing in its current form. Alternatively, I plan on being at PyCon2020 for the sprints and could be available then to work on it.
msg360730 - (view) Author: Dan Gass (dan.gass@gmail.com) Date: 2020-01-26 17:26
Sorry for not thinking of trying this sooner. Running garbage collection, import gc; gc.collect(), releases the resources and restores the expected reference counts.

From my perspective, this is satisfactory and could justify closing this bug report. 

The exception logic in Enum.__new__ could potentially be cleaned up for this use case, just not sure if that would introduce risk for other use cases. 

I am guessing (from the perspective of the larger Python exception handling design and implementation) what this bug report demonstrates had been previously thought through.
History
Date User Action Args
2020-02-10 04:26:22hongweipengsetnosy: + hongweipeng
2020-01-26 17:26:33dan.gass@gmail.comsetmessages: + msg360730
2020-01-25 21:27:05rhettingersetpriority: normal -> low
assignee: ethan.furman
2020-01-25 07:05:52xtreaksetnosy: + barry, eli.bendersky, ethan.furman
2020-01-25 06:30:33dan.gass@gmail.comcreate