This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients vstinner
Date 2021-06-14.21:26:28
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1623705988.63.0.204635618533.issue44422@roundup.psfhosted.org>
In-reply-to
Content
The threading.enumerate() code is simple:
---
# Active thread administration
_active_limbo_lock = _allocate_lock()
_active = {}    # maps thread id to Thread object
_limbo = {}

def enumerate():
    with _active_limbo_lock:
        return list(_active.values()) + list(_limbo.values())
---

values(), list() and list+list operations can call trigger a GC collection depending on the GC thresholds, created Python objects, etc.

During a GC collection, a Python object destructor (__del__) can again call threading.enumerate().

Problem: _active_limbo_lock is not re-entrant lock and so the second call hangs.

In practice, this issue was seen in OpenStack which uses eventlet, when a destructor uses the logging module. The logging module uses threading.current_thread(), but this function is monkey-patched by the eventlet module.

Extract of the eventlet function:
---
def current_thread():
    ...
    found = [th for th in __patched_enumerate() if th.ident == g_id]
    ...
---
https://github.com/eventlet/eventlet/blob/master/eventlet/green/threading.py

Attached PR makes _active_limbo_lock re-entrant to avoid this issue. I'm not sure if it's ok or not, I would appreciate to get a review and your feedback ;-)

Attached file triggers the threading.enumerate() hang on re-entrant call.
History
Date User Action Args
2021-06-14 21:26:28vstinnersetrecipients: + vstinner
2021-06-14 21:26:28vstinnersetmessageid: <1623705988.63.0.204635618533.issue44422@roundup.psfhosted.org>
2021-06-14 21:26:28vstinnerlinkissue44422 messages
2021-06-14 21:26:28vstinnercreate