This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: logging.config.dictConfig will convert namedtuple to ConvertingTuple
Type: crash Stage: resolved
Components: Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: hcoura, vinay.sajip, xtreak
Priority: normal Keywords: patch

Created on 2019-12-27 20:33 by hcoura, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
bug.py hcoura, 2019-12-28 15:48 Script to reproduce the bug
Pull Requests
URL Status Linked Edit
PR 17773 merged vinay.sajip, 2019-12-31 19:19
PR 17785 merged miss-islington, 2020-01-01 19:34
PR 17786 merged miss-islington, 2020-01-01 19:34
Messages (7)
msg358914 - (view) Author: Henrique (hcoura) Date: 2019-12-27 20:33
While passing

{
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "verbose": {"format": "%(levelname)s %(asctime)s %(module)s %(message)s"}
    },
    "handlers": {
        "stackdriver": {
            "class": "google.cloud.logging.handlers.CloudLoggingHandler",
            "client": client,
            "resource": resource,
        },
    },
    "root": {"level": "INFO", "handlers": ["stackdriver"]},
}

to logging.config.dictConfig it will convert resource, which is a namedtuple to ConvertingTuple, this will make google.cloud.logging.handlers.CloudLoggingHandler break down the line.

I am having to create a wrapper class like 

class Bla:
    resource = logging.resource.Resource(
        type="cloud_run_revision",
        labels={},
    )

    def _to_dict(self):
        return self.resource._to_dict()

to go around this
msg358920 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-28 01:01
Thanks for the report. Can you please attach a sample script with the handler used to illustrate the issue?
msg358948 - (view) Author: Henrique (hcoura) Date: 2019-12-28 15:48
Sorry for not sending a proper reproducible script when submitting the issue. End of the day and quite frustrated with the bug, anyway, I attached a full script that will show the error.

This is the custom handler I used:

class MyHandler(logging.StreamHandler):
    def __init__(self, resource, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.resource: namedtuple = resource
    
    def emit(self, record):
        record.msg += f" {self.resource.type}"
        return super().emit(record)


self.resource.type will throw and AttributeError when logging, because resource gets converted from a namedtuple to a ConvertingTuple.
msg358961 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-28 20:01
Thanks for the details. Attached is a complete script. Looking at the source code, anything that looks like a tuple is converted into a tuple. namedtuple produces factory function that itself is a subclass of tuple. It led me to this interesting issue over detecting if it's a namedtuple issue7796 where detecting _fields in msg134186 is described as a workaround. So logging might detect namedtuple and skip conversion to ConvertingTuple using the workaround but it appears more of the class of special casing it here in logging  that will depend on namedtuple internals. There are no test failures. I will leave it to vinay on this.

>>> import collections
>>> person = collections.namedtuple('Person', 'age')
>>> isinstance(person(age=20), tuple)
True
>>> print(person._source) # works on 3.6 but verbose and _source were removed in 3.7 with 8b57d7363916869357848e666d03fa7614c47897
from builtins import property as _property, tuple as _tuple
from operator import itemgetter as _itemgetter
from collections import OrderedDict

class Person(tuple):
    'Person(age,)'
// snip more source code for person

# Detect namedtuple by checking for _fields attribute

diff --git a/Lib/logging/config.py b/Lib/logging/config.py
index 4a3b8966ed..ba6937e725 100644
--- a/Lib/logging/config.py
+++ b/Lib/logging/config.py
@@ -448,7 +448,8 @@ class BaseConfigurator(object):
             value = ConvertingList(value)
             value.configurator = self
         elif not isinstance(value, ConvertingTuple) and\
-                 isinstance(value, tuple):
+                 isinstance(value, tuple) and\
+                 not hasattr(value, '_fields'):
             value = ConvertingTuple(value)
             value.configurator = self
         elif isinstance(value, str): # str for py3k
msg359166 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-01-01 19:32
New changeset 46abfc1416ff8e450999611ef8f231ff871ab133 by Vinay Sajip in branch 'master':
bpo-39142: Avoid converting namedtuple instances to ConvertingTuple. (GH-17773)
https://github.com/python/cpython/commit/46abfc1416ff8e450999611ef8f231ff871ab133
msg359167 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-01-01 20:06
New changeset 1d5a7e5694bd9104f56f4f28357c2d13afd58a29 by Vinay Sajip (Miss Islington (bot)) in branch '3.8':
bpo-39142: Avoid converting namedtuple instances to ConvertingTuple. (GH-17773) (GH-17785)
https://github.com/python/cpython/commit/1d5a7e5694bd9104f56f4f28357c2d13afd58a29
msg359168 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-01-01 20:07
New changeset 0e0e4acaabdcf8efb60cf75f08301022ac4422a4 by Vinay Sajip (Miss Islington (bot)) in branch '3.7':
bpo-39142: Avoid converting namedtuple instances to ConvertingTuple. (GH-17773) (GH-17786)
https://github.com/python/cpython/commit/0e0e4acaabdcf8efb60cf75f08301022ac4422a4
History
Date User Action Args
2022-04-11 14:59:24adminsetgithub: 83323
2020-01-01 20:27:28vinay.sajipsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-01-01 20:07:35vinay.sajipsetmessages: + msg359168
2020-01-01 20:06:55vinay.sajipsetmessages: + msg359167
2020-01-01 19:34:39miss-islingtonsetpull_requests: + pull_request17218
2020-01-01 19:34:09miss-islingtonsetpull_requests: + pull_request17217
2020-01-01 19:32:24vinay.sajipsetmessages: + msg359166
2019-12-31 19:23:58vinay.sajipsetversions: + Python 3.8, Python 3.9
2019-12-31 19:19:25vinay.sajipsetkeywords: + patch
stage: patch review
pull_requests: + pull_request17206
2019-12-28 20:01:14xtreaksetmessages: + msg358961
2019-12-28 15:48:23hcourasetfiles: + bug.py

messages: + msg358948
2019-12-28 01:01:12xtreaksetnosy: + vinay.sajip, xtreak
messages: + msg358920
2019-12-27 20:33:12hcouracreate