This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: logging: delay_fh option and configuration kwargs
Type: Stage:
Components: Library (Lib) Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: cdleary, vinay.sajip
Priority: normal Keywords: patch

Created on 2007-08-01 08:14 by cdleary, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
logging_delay_fh_patch.diff cdleary, 2007-08-01 08:14 StreamHandler stream callback function patch.
Messages (6)
msg52973 - (view) Author: Chris Leary (cdleary) Date: 2007-08-01 08:14
Streams (read: files) are opened in potentially destructive modes when loading logging configurations. This patch provides library users with a method of postponing the opening of streams. The method is simple, powerful, and should be fully compatible with existing infrastructure.

The StreamHandler now allows a callback function to be passed as an initialization parameter (@param strm). This function must be callable with no arguments and return a valid stream. A function passed in this manner can be initialized via the initialize_stream() method call; however, if the user chooses not to call initialize_stream() explicitly, the stream is initialized on the first attempt to emit a record.

To take advantage of this ability, the FileHandler has been extended with callback-passing functionality. If the delay_fh initialization parameter is asserted, a file-opening closure is passed to the StreamHandler (superclass) initialization method.

This ability has also been extended to the module's fileConfig() function. Though fileConfig()'s optional delay_fh parameter, the user may load a configuration from file without undesired file handle opening effects. This is particularly useful when running many instances of the same program on a single file system -- the user now has the ability to change (at runtime) FileHandlers that would otherwise be opened in 'w' mode, clobbering the other instantiation's logs.

Another, seemingly tangential portion of the patch adds a kwargs specifier to the configuration file. This is to add the flexibility of adding a delay_fh keyword indicator without specifying other arguments unnecessarily as part of 'args'. It still follows the 1.5 compatible apply() syntax, and it only takes another 4 lines ;)

Implementation Notes:
- Tested in regtest and with an application I wrote that's quite logging-intensive, and both check out OK so far.
- I broke some functions I was working on into helpers for readability, as they were getting too long. They're all initialization functions that shouldn't be performance critical.
- Private functions that I added weren't given default values for the delay_fh parameter, since nobody knows about them; however, I gave the private functions that were already there a default value for the delay_fh parameter just to be extra safe and backwards compatible. Hopefully nobody's using those anyway.
- I tried my best to follow existing style -- feel free to yell at me if I missed it somewhere :D

- Chris Leary
msg55747 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2007-09-08 12:01
Thanks for the patch. I think it would be simpler to just implement an
optional delay parameter in FileHandler and its subclasses: then the
configuration stuff need not be touched at all, since the delay
parameter can be specified in the configuration file. There would then
be no need for a callback or a file-opening closure: If delay is False
(the default), FileHandler would behave as it does now. If True, then
the open code in FileHandler's constructor would be deferred until the
time emit() was called on the instance.

Would this meet your needs?
msg55772 - (view) Author: Chris Leary (cdleary) Date: 2007-09-10 01:28
Hi Vinay,

I was actually trying to address a use case where the delay_fh option in
the fileConfig() would be necessary.

Let's say I'm running a simulator that I run many instances of at once.
The logging configuration is extensive, so I want to use a configuration
file; however, I don't want any existing log files to be clobbered as
soon as I run fileConfig() -- I want to run fileConfig() to load the
configuration, then remove/modify the handlers /before/ they touch the
file handles.

If fileConfig has no delay_fh option, fileConfig() would create the
FileHandlers with delay_fh as False; therefore, adding the option to the
FileHandler alone isn't enough to fix this use case.

Let me know if this is unclear, or if I should provide a more concrete
example.
msg55776 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2007-09-10 08:54
I'm not sure I agree with "If fileConfig has no delay_fh option,
fileConfig() would create the FileHandlers with delay_fh as False".
fileConfig will create the FileHandlers using the args= section entry to
pass to the constructor. Hence, by specifying True for this parameter,
you get the FileHandlers to delay opening the files. In fact you get
more control - you can specify True for some FileHandlers and False for
others.

The relevant code is in config.py in _install_handlers:

...
klass = eval(klass, vars(logging))
args = cp.get(sectname, "args")
args = eval(args, vars(logging))
h = apply(klass, args)
...
msg55780 - (view) Author: Chris Leary (cdleary) Date: 2007-09-10 14:26
Ah, I see what the miscommunication is now.

Yes, implementing the feature as you suggested would give the user the
ability to specify the delayed handle-opening /within/ the configuration
file. What I had intended was a way to load/inspect a file configuration
without it having any handle-opening side affects, even if it were a
totally unknown configuration file.

If you want to assume a good trust relationship with the contents of a
file configuration that you're loading (as in, you trust that it won't
clobber any files that you still want), then that works well. I just
assumed it would be useful to open a file configuration in an
"untrusted" way (via delay_fh) in order to inspect/modify it. It seems
to me to put more power in the hands of the loader of the configuration,
rather than leaving the loader subject to the potentially "bad" contents
of the configuration file.

If you think I'm being too paranoid or feel that this is too obscure a
use case, feel free to implement it how you see fit! :)
msg61628 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2008-01-24 12:44
Added optional delay argument to FileHandler and subclasses.
History
Date User Action Args
2022-04-11 14:56:25adminsetgithub: 45265
2008-01-24 12:44:52vinay.sajipsetstatus: open -> closed
resolution: fixed
messages: + msg61628
2007-09-10 14:26:47cdlearysetmessages: + msg55780
2007-09-10 08:54:44vinay.sajipsetmessages: + msg55776
2007-09-10 01:28:25cdlearysetmessages: + msg55772
2007-09-08 12:01:46vinay.sajipsetmessages: + msg55747
2007-08-01 08:14:34cdlearycreate