Author vinay.sajip
Recipients socketpair, vinay.sajip
Date 2011-04-29.16:48:25
SpamBayes Score 1.98175e-14
Marked as misclassified No
Message-id <1304095706.72.0.567265130559.issue11950@psf.upfronthosting.co.za>
In-reply-to
Content
I had to delete my previous response to the initial post, as something got mangled in it. Here's what I meant to say:
----------------------------------------------
It sounds to me as if there's a problem in yum that needs sorting out: I'm not sure why it would need dozens of loggers.

As your workaround snippet shows, leaking files is not due to loggers but due to handlers - they're a different animal. Handlers need not be leaked - they can just be closed when you're done with them. Handlers *are* stored with weak references.

The design contract for loggers precludes using weak references for them. For example, if I use a library and from my logging initialization code get a reference to its top-level logger and e.g. set that logger's level, then when I've finished, there may not be any other references to that logger until I do an import of some library module. I wouldn't want the reference to the logger to go away, because then the library wouldn't know my intention regarding the level setting.

Normal practice would be to add a few handlers to yum's top-level logger and let descendant loggers inherit them. I'm not familiar with yum, so I don't know if/why it deviates from this common and best practice approach.

As you state, replacing {} with WeakValueDictionary() wouldn't solve the problem, and would violate the design assumptions/contract relating to loggers.

By the way, the logging module does not store loggers in lists and dicts - just in a single dict in a manager. Handlers are stored in weak reference dicts as well as lists but that does not mean they are guaranteed to be garbage collected, since there could also be references to them in user code. Likewise for loggers - typically there would be references to them in lines like

logger = logging.getLogger(__name__)

in numerous modules. Even if yum consists of dozens of modules (which perhaps explains dozens of loggers), they needn't leak files, as just adding a single FileHandler to yum's top-level logger would serve to capture logging events from the entire yum package hierarchy.
History
Date User Action Args
2011-04-29 16:48:26vinay.sajipsetrecipients: + vinay.sajip, socketpair
2011-04-29 16:48:26vinay.sajipsetmessageid: <1304095706.72.0.567265130559.issue11950@psf.upfronthosting.co.za>
2011-04-29 16:48:25vinay.sajiplinkissue11950 messages
2011-04-29 16:48:25vinay.sajipcreate