This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author benspiller
Recipients benspiller
Date 2018-11-07.17:43:56
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1541612637.38.0.788709270274.issue35185@psf.upfronthosting.co.za>
In-reply-to
Content
I just came across a fairly serious thread-safety / race condition bug in the logging.Loggers class, which causes random log lines to be lost i.e. not get passed to some of the registered handlers, if (other, unrelated) handlers are being added/removed using add/removeHandler from another thread during logging. This potentially affects all log handler classes, though for timing reasons I've found it easiest to reproduce with the logging.FileHandler. 

See attached test program that reproduces this. 

I did some debugging and looks like although add/removeHandler are protected by _acquireLock(), they modify the self.handlers list in-place, and the callHandlers method iterates over self.handlers with no locking - so if you're unlucky you can end up with some of your handlers not being called. 

A trivial way to fix the bug is by editing callHandlers and copying the list before iterating over it:
- for hdlr in c.handlers:
+ for hdlr in list(c.handlers):

However since that could affect the performance of routine log statements a better fix is probably to change the implementation of add/removeHandler to avoid in-place modification of self.handlers so that (as a result of the GIL) it'll be safe to iterate over the list in callHandlers, e.g. change removeHandler like this:

- self.handlers.remove(hdlr)
+ newhandlers = list(self.handlers)
+ newhandlers.remove(hdlr)
+ self.handlers = hdlr

(and the equivalent in addHandler)
History
Date User Action Args
2018-11-07 17:43:57benspillersetrecipients: + benspiller
2018-11-07 17:43:57benspillersetmessageid: <1541612637.38.0.788709270274.issue35185@psf.upfronthosting.co.za>
2018-11-07 17:43:57benspillerlinkissue35185 messages
2018-11-07 17:43:57benspillercreate