This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: "Logging HOWTO" should share an example of best practices for using logging in a library
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: Nathaniel Manista, docs@python, pms.coder, rhettinger, tlesher, vinay.sajip
Priority: normal Keywords:

Created on 2018-09-05 20:39 by Nathaniel Manista, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (7)
msg324652 - (view) Author: Nathaniel Manista (Nathaniel Manista) Date: 2018-09-05 20:39
https://docs.python.org/3.8/howto/logging.html#configuring-logging-for-a-library is a bit too short and doesn't answer some questions that I have as a library author about the best ways to use logging in a library.

Should I make use of a single logger object in my library, multiple loggers in a tree, or multiple unrelated loggers? Since I have just one public module, I'm tempted to say that I should just use one logger object.

Should I present as part of my public API the name of the logger object(s) used? Probably - the documentation starts with "When developing a library which uses logging, you should take care to document how the library uses logging - for example, the names of loggers used.". But should I also include the logger objects in my API? If an application wants to reach my logger, why should they use "logging.getLogger(<name that I shared with them>)" rather than "my_library.LOGGER"?

Should I use my library's fully-qualified name as the name of its logger? This seems like a great idea because it's unlikely to bring up any new conflicts. Is it a best practice? Is there any better practice?

The "Configuring Logging for a Library" text could answer these questions, and maybe it should, but... I really think a toy library example that library authors could follow would also help a great deal.
msg324660 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-09-06 04:54
FWIW, I think the practices vary somewhat widely and that none of your questions have generally agreed upon answers.  Part of the reason that there are so many ways to go is that the package was modeled after Java APIs where the practices and needs were also widely varied.  Given the absence of clear-cut best practices, the docs should probably defer to StackOverflow and various blog posts.
msg324711 - (view) Author: Nathaniel Manista (Nathaniel Manista) Date: 2018-09-06 19:17
Something... related, that may perhaps belong in a separate issue, but that I want to at least mention here because it would be solved if logging-in-libraries best practices were authoritatively documented and exemplified: it's just too consarn easy to "hold [the logging module] wrong" and wind up with "No handlers could be found for logger" log spam: https://www.google.ca/search?q=site:stackoverflow.com+"No+handlers+could+be+found+for+logger". Why should a Logger object need to have .basicConfig() called on it after retrieval (where "retrieval" means "getLogger call") and before use anyway? Why can't it just be ready for use (at least as ready for use as .basicConfig makes it) when passed from the logging module to the logging-using module like nearly any other object passed from the standard library to standard-library-using code?

(The documentation says "No handlers could be found for logger" spam only happens pre-3.2, but some of my users at least think they see it in later Pythons.)
msg324726 - (view) Author: (pms.coder) * Date: 2018-09-07 08:04
I would be happy to see such HOWTO, so we could relay on actual documentation instead of the "Internet".

I think the best approach would be to get those 10-20 questions you ( Nathaniel Manista) just asked and post them to python-dev mailing list, so senior developers/committers could answer them. 

Then we could establish the "gold standard" for logging and write about it in the HOWTO.
msg324727 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-09-07 08:13
> Then we could establish the "gold standard" for logging and write 
> about it in the HOWTO.

There really isn't a gold standard.  There are many widely varied logging practices, each adapted to the needs of the application.  The module is complicated primarily because there are so many legitimate techniques and reasons for using those techniques.

> (The documentation says "No handlers could be found for logger" 
> spam only happens pre-3.2, but some of my users at least think
> they see it in later Pythons.)

If you find an actual error in the docs, please make a specific report so that it can be confirmed and fixed.
msg324745 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2018-09-07 14:47
> Should I make use of a single logger object in my library, multiple loggers
> in a tree, or multiple unrelated loggers? Since I have just one public module,
> I'm tempted to say that I should just use one logger object.

Yes. The advanced logging tutorial (part of the HOWTO) states:

"A good convention to use when naming loggers is to use a module-level logger, in each module which uses logging, named as follows:

logger = logging.getLogger(__name__)

This means that logger names track the package/module hierarchy, and it’s intuitively obvious where events are logged just from the logger name."

This seems clear enough to me. Did you look at the whole of the HOWTO? If not, I'd advise you to take the time

> (The documentation says "No handlers could be found for logger" 
> spam only happens pre-3.2, but some of my users at least think
> they see it in later Pythons.)

In 3.2, the change was made to introduce a "handler of last resort" instead of the "no handlers could be found" message, which is only printed if logging.lastResort is set to None or other "false" value (by default, it's set to a console handler at WARNING level emitting to sys.stderr). If you can give specifics of a later Python version where this happens (precise Python version, OS etc.) then we can address it.

Practices vary widely because use cases for logging are widely varied. A simple one-off script has different needs from a product with a long shelf-life, a command-line application has different needs from a long-running server application, a single-author application has different needs from one which is put together from lots of different dependencies by different authors, etc.

> Why should a Logger object need to have .basicConfig() called on
> it after retrieval (where "retrieval" means "getLogger call") and
> before use anyway?

It doesn't. If you want output, you need handlers. The basicConfig() API is just one way of attaching handlers to loggers for simple use cases. The question of handlers is also no concern of a library author, other than the NullHandler part, as the application is where logging should be configured (as mentioned in the "Configuring Logging for a Library" section).

> Then we could establish the "gold standard" for logging and write
> about it in the HOWTO.

There's no one size fits all, I'm afraid, because the package is designed to cover a wide range of use cases, not just the simplest ones. While there is (IMO) comprehensive documentation, specific proposals for improvements will be looked at constructively.
msg327850 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2018-10-17 01:42
Closing, as no specific proposals received.
History
Date User Action Args
2022-04-11 14:59:05adminsetgithub: 78771
2018-10-17 01:42:04vinay.sajipsetstatus: open -> closed
resolution: not a bug
messages: + msg327850

stage: resolved
2018-09-07 14:47:03vinay.sajipsetmessages: + msg324745
2018-09-07 08:13:22rhettingersetversions: - Python 2.7, Python 3.4, Python 3.5, Python 3.6
nosy: + vinay.sajip

messages: + msg324727

assignee: docs@python -> vinay.sajip
2018-09-07 08:04:41pms.codersetnosy: + pms.coder
messages: + msg324726
2018-09-06 19:17:38Nathaniel Manistasetmessages: + msg324711
2018-09-06 15:49:54tleshersetnosy: + tlesher
2018-09-06 04:54:17rhettingersetnosy: + rhettinger
messages: + msg324660
2018-09-05 20:39:26Nathaniel Manistacreate