Author ncoghlan
Recipients JohanAR, davin, itamarst, ncoghlan, python-dev, rhettinger, sbt, tim.peters, yselivanov, zzzeek
Date 2017-08-17.13:52:08
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1502977929.37.0.818258779028.issue14976@psf.upfronthosting.co.za>
In-reply-to
Content
Itamar wrote up a post describing the GC variant of this problem in more detail: https://codewithoutrules.com/2017/08/16/concurrency-python/

In particular, he highlighted a particularly nasty action-at-a-distance variant of the deadlock where:

1. Someone registers a logging.handlers.QueueHandler instance with the logging system
2. One or more types in the application or libraries it uses call logging functions in a __del__ method or a weakref callback
3. A GC cycle triggers while a log message is already being processed and hence the thread already holds the queue's put() lock
4. Things deadlock because the put() operation isn't re-entrant

As far as I can see, there's no application level way of resolving that short of "Only register logging.handlers.QueueHandler with a logger you completely control and hence can ensure is never used in a __del__ method or weakref callback", which doesn't feel like a reasonable restriction to place on the safe use of a standard library logging handler.
History
Date User Action Args
2017-08-17 13:52:09ncoghlansetrecipients: + ncoghlan, tim.peters, rhettinger, zzzeek, python-dev, sbt, JohanAR, yselivanov, itamarst, davin
2017-08-17 13:52:09ncoghlansetmessageid: <1502977929.37.0.818258779028.issue14976@psf.upfronthosting.co.za>
2017-08-17 13:52:09ncoghlanlinkissue14976 messages
2017-08-17 13:52:08ncoghlancreate