classification
Title: Any logging causes assertLogs to pass
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: iritkatriel, miss-islington, pitrou, udalrich.schermer, vinay.sajip
Priority: normal Keywords: patch

Created on 2020-09-30 19:13 by udalrich.schermer, last changed 2020-10-05 14:54 by iritkatriel. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22526 merged iritkatriel, 2020-10-03 16:57
PR 22537 merged miss-islington, 2020-10-04 13:16
PR 22538 closed miss-islington, 2020-10-04 13:16
PR 22540 merged miss-islington, 2020-10-04 13:28
PR 22565 merged iritkatriel, 2020-10-05 14:38
Messages (9)
msg377719 - (view) Author: Troy Daniels (udalrich.schermer) Date: 2020-09-30 19:13
The following test code unexpectedly passes.

import logging
import unittest

LOG_FORMAT = '%(levelname)-10s %(asctime)s: %(message)s'


def set_up_logger(app_name, level=logging.INFO, file="test.log"):
    formatter = logging.Formatter(LOG_FORMAT)
    log = logging.getLogger(app_name)
    # The next line lets the test pass
    log.setLevel(level)
    return log

logger = set_up_logger(__name__)

class TestLogging(unittest.TestCase):
    def test_logging(self):
        with self.assertLogs(level=logging.WARNING):
            logger.info('foo')

Based on discussion at https://stackoverflow.com/questions/64141681/any-level-of-logging-lets-assertlogs-pass, it appears that I need to pass in the logger which is going to be doing the logging.  

Doing so would make the test over-specific and fragile.  The requirement is that a warning be logged, not that a specific logger issue the warning.

This was verified with Python 3.8.5 on a Mac, but probably exists in most/all other versions.
msg377880 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-10-03 13:00
You didn't specify the logger in the assertLogs() call. 

The documentation says that "The default is the root logger, which will catch all messages."

https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertLogs

This is behaving as expected.
msg377938 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-10-04 13:16
New changeset 1ed54435268b285964141fb74d47ceaa33ea79ab by Irit Katriel in branch 'master':
bpo-41898: add caveat on root logger seeing all messages in assertLogs doc (GH-22526)
https://github.com/python/cpython/commit/1ed54435268b285964141fb74d47ceaa33ea79ab
msg377939 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-10-04 13:27
New changeset b2611fac05ed391d68f09781903aae7394fab832 by Miss Skeleton (bot) in branch '3.8':
bpo-41898: add caveat on root logger seeing all messages in assertLogs doc (GH-22526) (GH-22537)
https://github.com/python/cpython/commit/b2611fac05ed391d68f09781903aae7394fab832
msg377946 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-10-04 15:38
New changeset faa8c6a8f1fe9c3bac31061e8a59a686b983ebb8 by Miss Skeleton (bot) in branch '3.9':
bpo-41898: add caveat on root logger seeing all messages in assertLogs doc (GH-22526) (GH-22540)
https://github.com/python/cpython/commit/faa8c6a8f1fe9c3bac31061e8a59a686b983ebb8
msg377951 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-10-04 15:51
I'm not sure my patch completely resolved this issue. I'm adding Antoine re the OP's question about the assertLogs API. 

> Doing so would make the test over-specific and fragile.  The requirement is > that a warning be logged, not that a specific logger issue the warning.

My view is that your test is fragile because it just asks "was anything logged" without checking the log message. The docs show how to verify that the log message is what you expect it to be.
msg377957 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-10-04 16:16
Troy, 

I'm also confused about how you would have liked assertLogs to behave you say:

1. I don't want to specify the logger because I don't care which logger the message comes from.

2. I don't want it to catch messages from other loggers (whatever other means here).  But -- see 1 -- you didn't specify the logger because you don't care which logger.

What do you expect assertLogs to do when you don't specify a logger?
msg378030 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-10-05 14:15
Please reopen this issue. I understand after a discussion on stackoverflow what the problem is, and there is indeed a bug. I will create a PR with a unit test for it an a fix.
msg378039 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-10-05 14:54
Strike that - Troy has created the new issue 41943 so this one can remain closed.
History
Date User Action Args
2020-10-05 14:54:57iritkatrielsetmessages: + msg378039
2020-10-05 14:38:07iritkatrielsetpull_requests: + pull_request21560
2020-10-05 14:15:36iritkatrielsetmessages: + msg378030
2020-10-04 16:16:07iritkatrielsetmessages: + msg377957
2020-10-04 15:51:13iritkatrielsetnosy: + pitrou
messages: + msg377951
2020-10-04 15:38:47vinay.sajipsetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.9, Python 3.10
2020-10-04 15:38:09vinay.sajipsetmessages: + msg377946
2020-10-04 13:28:49miss-islingtonsetpull_requests: + pull_request21543
2020-10-04 13:27:21vinay.sajipsetmessages: + msg377939
2020-10-04 13:16:30miss-islingtonsetpull_requests: + pull_request21541
2020-10-04 13:16:21miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request21540
2020-10-04 13:16:16vinay.sajipsetnosy: + vinay.sajip
messages: + msg377938
2020-10-03 16:57:41iritkatrielsetkeywords: + patch
stage: patch review
pull_requests: + pull_request21531
2020-10-03 13:00:20iritkatrielsetnosy: + iritkatriel
messages: + msg377880
2020-09-30 19:13:37udalrich.schermercreate