This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Marc.Abramowitz
Recipients Marc.Abramowitz, SilentGhost, lukasz.langa
Date 2016-04-07.15:41:54
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1460043714.38.0.358477782064.issue26710@psf.upfronthosting.co.za>
In-reply-to
Content
Some more info on the logging example I gave.

So here is a program called `my_app.py`:

```
import os
import logging.config

logging.config.fileConfig('logging.ini', defaults=os.environ)
logger = logging.getLogger(__name__)
logger.debug('debug msg')
logger.info('info msg')
logger.warn('warn msg')
logger.error('error msg')
root_logger = logging.getLogger()
print('root_logger.level = %d; logging.WARN = %d; logging.DEBUG = %d'
      % (root_logger.level, logging.WARN, logging.DEBUG))
```

Note that it calls logging.config.fileConfig with defaults=os.environ so that environment variables can be used to affect the logging configuration.


And here is `logging.ini`:

```
###
# logging configuration
# http://docs.pylonsproject.org/projects/pyramid/en/1.7-branch/narr/logging.html
###

[loggers]
keys = root, fakeproject, sqlalchemy

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = %(logging_logger_root_level)s
handlers = console

[logger_fakeproject]
level = DEBUG
handlers =
qualname = fakeproject

[logger_sqlalchemy]
level = INFO
handlers =
qualname = sqlalchemy.engine
# "level = INFO" logs SQL queries.
# "level = DEBUG" logs SQL queries and results.
# "level = WARN" logs neither.  (Recommended for production systems.)

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s
```

Note that in the `logger_root` section of `logging.ini`, the variable `logging_logger_root_level` is referenced but not defined. For now, we are going to rely on getting that from an environment variable.


If I provide an environment variable when running the program:

```
$ LOGGING_LOGGER_ROOT_LEVEL=DEBUG python my_app.py
2016-04-07 08:26:36,184 DEBUG [__main__:6][MainThread] debug msg
2016-04-07 08:26:36,184 INFO  [__main__:7][MainThread] info msg
2016-04-07 08:26:36,184 WARNI [__main__:8][MainThread] warn msg
2016-04-07 08:26:36,184 ERROR [__main__:9][MainThread] error msg
root_logger.level = 10; logging.WARN = 30; logging.DEBUG = 10
```

then it works and the root logger level is DEBUG as expected. Great!

But what happens if the user leaves out the environment variable?

```
$ python my_app.py
Traceback (most recent call last):
  File "my_app.py", line 4, in <module>
    logging.config.fileConfig('logging.ini', defaults=os.environ)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/config.py", line 86, in fileConfig
    _install_loggers(cp, handlers, disable_existing_loggers)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/config.py", line 196, in _install_loggers
    level = cp.get(sectname, "level")
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 623, in get
    return self._interpolate(section, option, value, d)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 669, in _interpolate
    option, section, rawval, e.args[0])
ConfigParser.InterpolationMissingOptionError: Bad value substitution:
	section: [logger_root]
	option : level
	key    : logging_logger_root_level
	rawval : %(logging_logger_root_level)s
```

An error occurs as expected. 

But I'd like to be able to provide a default value so that user doesn't have to set the environment variable, so let's add this to the top of `logging.ini`:

```
[DEFAULT]
logging_logger_root_level = WARN
```

Now let's run the program again without the environment variable to see if that fixed the problem:

```
$ python my_app.py
2016-04-07 08:33:07,101 WARNI [__main__:8][MainThread] warn msg
2016-04-07 08:33:07,101 ERROR [__main__:9][MainThread] error msg
root_logger.level = 30; logging.WARN = 30; logging.DEBUG = 10
```

Awesome! It worked and set the root logger level to the default of WARN, as expected.

Now what happens if we try to override the default with an environment variable?

```
$ LOGGING_LOGGER_ROOT_LEVEL=DEBUG python my_app.py
2016-04-07 08:33:56,047 WARNI [__main__:8][MainThread] warn msg
2016-04-07 08:33:56,048 ERROR [__main__:9][MainThread] error msg
root_logger.level = 30; logging.WARN = 30; logging.DEBUG = 10
```

Doh! The root logger level is still WARN. So the default in the ini file took precedence over what the user provided. This is unfortunate.

So how does one provide defaults while also allowing overriding those defaults?
History
Date User Action Args
2016-04-07 15:41:54Marc.Abramowitzsetrecipients: + Marc.Abramowitz, SilentGhost, lukasz.langa
2016-04-07 15:41:54Marc.Abramowitzsetmessageid: <1460043714.38.0.358477782064.issue26710@psf.upfronthosting.co.za>
2016-04-07 15:41:54Marc.Abramowitzlinkissue26710 messages
2016-04-07 15:41:54Marc.Abramowitzcreate