Title: Provide logging.config.configParserConfig
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.4
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: python-dev, r.david.murray, thbach, vinay.sajip
Priority: normal Keywords: patch

Created on 2012-10-02 15:27 by thbach, last changed 2014-03-10 22:11 by python-dev. This issue is now closed.

File name Uploaded Description Edit
configParserConfig.patch thbach, 2012-10-02 15:27 patch to provide functionality review
Messages (8)
msg171812 - (view) Author: Thomas Bach (thbach) Date: 2012-10-02 15:27
Currently logging.config provides a fileConfig function which reads a ini-style file via configparser.ConfigParser. I would like to have a function e.g. configParserConfig which accepts a ConfigParser instance and configures logging directly from the settings found in there. The main reasons for this are:

1) I think it is rather common for an application that provides an interface to configure its logging via an ini file to use this ini file also for further application configuration. With the current implementation the file is read twice and ConfigParser is initialized two times.

2) Currently it is not idiomatic how to alter an ini-file configuration e.g. by options passed in via command-line. The new function provides a clear solution: create a ConfigParser instance, parse the ini file, alter the configuration and pass it on to logging.config.configParserConfig.

In fact, the new functionality is easy to achieve by refactoring logging.config a bit (see attached patch).
msg171827 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-10-02 19:23
Thanks for the suggestion - I'm sorry, but I'm not inclined to add this. My reasoning is as follows:

1. I want to encourage usage of the dictConfig() API, as fileConfig() does not cover as much of the logging API as dictConfig() does (for example, Filters). I'd like to minimise the maintenance I have to do for fileConfig()-related code, and would prefer not to add any auxiliary APIs around fileConfig().
2. The file reading time and ConfigParser instantiation time are not likely to be a performance problem in practice, as logging  configuration is an infrequent operation (a one-off operation in most cases).
3. You can also pass in a file-like object rather than a filename, and in that case, fileConfig() will use readfp() if available. While you might have to seek to the beginning of the file to pass it to another ConfigParser instance, that is likely to be just a pointer adjustment in a buffer rather than actual disk I/O (config files are usually pretty small).

I hope you will agree. I'll leave the issue as pending for now, and close it in a day or two.
msg171828 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-10-02 19:31
Vinay, you missed one use case in his request: reading the program's configuration, *modifying it* (based on command line args), and then passing it to logging.  How would you suggest he handle that use case?  Is there an easy way to get from a loaded configuration file to a dictionary for use in dictConfig?
msg171996 - (view) Author: Thomas Bach (thbach) Date: 2012-10-04 21:11
vinay: I understand your preference of dictConfig over fileConfig as maintainer. But as an application developer I want to provide my user an easy way to adjust logging herself. In the end of the day she is the one knowing what has to be logged in which place. Therefor, having something like fileConfig is essential.

david: The modified configuration can be passed to fileConfig via a StringIO instance. 

The downside is that ConfigParser instances with e.g. 'allow_no_value' set to True are currently difficult to handle – e.g.:

>>> sys.version
'3.2.3 (default, Jun 25 2012, 23:10:56) \n[GCC 4.7.1]'
>>> cp = configparser.ConfigParser(allow_no_value=True)
>>> cp.read_string('[foo]\nbar')
>>> buf = io.StringIO()
>>> cp.write(buf)
>>> logging.config.fileConfig(buf)
ParsingError                              Traceback (most recent call last)
<ipython-input-67-0717fe665796> in <module>()
----> 1 logging.config.fileConfig(buf)

/usr/lib/python3.2/logging/ in fileConfig(fname, defaults, disable_existing_loggers)
     64     cp = configparser.ConfigParser(defaults)
     65     if hasattr(fname, 'readline'):
---> 66         cp.read_file(fname)
     67     else:

/usr/lib/python3.2/ in read_file(self, f, source)
    706             except AttributeError:
    707                 source = '<???>'
--> 708         self._read(f, source)
    710     def read_string(self, string, source='<string>'):

/usr/lib/python3.2/ in _read(self, fp, fpname)
   1079         # if any parsing errors occurred, raise an exception
   1080         if e:
-> 1081             raise e
   1082         self._join_multiline_values()

ParsingError: Source contains parsing errors: <???>
	[line  2]: 'bar\n'

Hence, logging.config.fileConfig should at least provide a way to pass in arguments for the ConfigParser initialization. Anyways, I think it is much cleaner to provide a function which gets the configuration directly from the ConfigParser instance.
msg172015 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-10-04 22:50
I could consider relaxing the parameters on fileConfig such that instead of accepting just a string or a file-like object, it additionally accepts a ConfigParser instance. More specifically:

def fileConfig(file_or_fname_or_cp, defaults=None):
    if isinstance(file_or_fname_or_cp, RawConfigParser):
        cp = file_or_filename_or_cp
        cp = ConfigParser.ConfigParser(defaults)
        if hasattr(cp, 'readfp') and\
            hasattr(file_or_fname_or_cp, 'readline'):

    formatters = _create_formatters(cp)

This will only require (in addition to the above) small tweaks to docs
and tests. It would appear to fit the bill for your use case. Do you agree?
msg172193 - (view) Author: Thomas Bach (thbach) Date: 2012-10-06 12:56
Yeah, the change you suggest sounds reasonable.

Thanks for reconsidering the case!
msg172459 - (view) Author: Roundup Robot (python-dev) Date: 2012-10-09 08:06
New changeset ce0d0d052494 by Vinay Sajip in branch 'default':
Closes #16110: fileConfig now accepts a pre-initialised ConfigParser instance.
msg213103 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-10 22:11
New changeset 113341605247 by R David Murray in branch 'default':
whatsnew: logging.fileConfig accepts ConfigParser instances. (#16110)
Date User Action Args
2014-03-10 22:11:26python-devsetmessages: + msg213103
2012-10-09 08:06:43python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg172459

resolution: fixed
stage: resolved
2012-10-06 12:56:05thbachsetmessages: + msg172193
2012-10-04 22:53:04vinay.sajipsetresolution: wont fix -> (no value)
2012-10-04 22:50:42vinay.sajipsetmessages: + msg172015
2012-10-04 21:11:12thbachsetmessages: + msg171996
2012-10-02 19:31:48r.david.murraysetstatus: pending -> open
nosy: + r.david.murray
messages: + msg171828

2012-10-02 19:23:41vinay.sajipsetstatus: open -> pending
assignee: vinay.sajip
resolution: wont fix
messages: + msg171827
2012-10-02 16:09:22r.david.murraysetnosy: + vinay.sajip

type: enhancement
versions: + Python 3.4
2012-10-02 15:27:11thbachcreate