classification
Title: ConvertingList and ConvertingTuple lack iterators and ConvertingDict lacks items()
Type: enhancement Stage: patch review
Components: Library (Lib) Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: godot_gildor, mivade, python-dev, serhiy.storchaka, vinay.sajip
Priority: normal Keywords: patch

Created on 2020-06-30 23:10 by godot_gildor, last changed 2020-07-31 08:58 by vinay.sajip.

Pull Requests
URL Status Linked Edit
PR 21248 open python-dev, 2020-06-30 23:40
Messages (12)
msg372724 - (view) Author: Brett Hannigan (godot_gildor) * Date: 2020-06-30 23:10
The logging.config module uses three internal data structures to hold items that may need to be converted to a handler or other object: ConvertingList, ConvertingTuple, and ConvertingDict.

These three objects provide interfaces to get converted items using the __getitem__ methods. However, if a user tries to iterate over items in the container, they will get the un-converted entries.
msg372731 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-07-01 06:05
This is a change in behaviour, so probably needs to be added to future versions only.
msg372883 - (view) Author: Michael DePalatis (mivade) * Date: 2020-07-02 20:54
I think the other issue here is that the ConvertingX classes aren't documented apart from comments in the code where they are defined.
msg372996 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-07-04 21:16
> I think the other issue here is that the ConvertingX classes aren't documented apart from comments in the code where they are defined.

That's deliberate - they're considered an internal implementation detail.
msg372999 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-07-04 21:32
If you are going to make them public general purpose classes you need to implement also support of slices, __reversed__(), index(), remove(), count(), sort(), copy() in ConvertingList and more methods in ConvertingTuple, and ConvertingDict.

If it is not goal then I think that no any changes are needed. If they are internal classes they needed only features which are used internally by the module code.
msg373120 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-07-06 14:29
> If it is not goal

I don't have a goal to make these part of a documented API. OP, can you share a use case where you need to iterate over these internal structures?
msg373127 - (view) Author: Brett Hannigan (godot_gildor) * Date: 2020-07-06 14:59
I encountered the need for the iterators when trying to create a subclass of the QueueHandler class that would manage both the QueueHandler and the QueueListener. The implementation is very similar to that described in this Medium post:
https://medium.com/@rob.blackbourn/how-to-use-python-logging-queuehandler-with-dictconfig-1e8b1284e27a

Both the original poster and I encountered one small issue: when using a dictConfig to instantiate the new subclass, the main QueueHandler gets a ConvertingList of the handlers that the user has requested be used. The subclass would then pass these to the QueueListener, but the constructor for the QueueListener takes *handlers (that is, it will convert the ConvertingList to a tuple). Unfortunately, because ConvertingList does not expose the iterator, converting from the ConvertingList to the tuple results in a tuple of unconverted handler references (ultimately strings).

The author of the Medium article gets around this by creating a little function that simply loops over the length of the ConvertingList and does a "get" on each item on the list, to ensure that the item is converted. Since ConvertingList is not documented though, there is concern that this approach could break in the future if the interface changes etc. 

With the implementation of the iterator in this PR, the conversion of the ConvertingList to the tuple will automatically result in a tuple of converted handlers, so one doesn't need to know about the ConvertingList object - it handles things behind the scenes.

Here is the code that the Medium article currently uses to force conversion:

def _resolve_handlers(l):
    if not isinstance(l, ConvertingList):
        return l

    # Indexing the list performs the evaluation.
    return [l[i] for i in range(len(l))]
msg373133 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-07-06 15:51
OK, seems like a reasonable use case. I haven't looked at the PR yet, as it still has a "CLA not signed" label, and I normally wait until the CLA is signed before looking more closely at PRs.
msg373135 - (view) Author: Brett Hannigan (godot_gildor) * Date: 2020-07-06 15:54
Thanks.

I don't know why it still says CLA not signed - I signed it a week ago, but I'll try to figure that out this week.
msg373184 - (view) Author: Brett Hannigan (godot_gildor) * Date: 2020-07-06 22:04
O.K. CLA is now signed and if I check on the "check-yourself" with my github user it is showing that I have signed it now.
msg374604 - (view) Author: Brett Hannigan (godot_gildor) * Date: 2020-07-30 07:58
Just wanted to check-in to see if there were any updates on my proposed PR?
msg374631 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-07-31 08:58
Thanks for the PR. I reviewed it and requested changes about 3 weeks ago - you should have received a notification from GitHub when that happened.
History
Date User Action Args
2020-07-31 08:58:16vinay.sajipsetmessages: + msg374631
2020-07-30 07:58:26godot_gildorsetmessages: + msg374604
2020-07-06 22:04:36godot_gildorsetmessages: + msg373184
2020-07-06 15:54:35godot_gildorsetmessages: + msg373135
2020-07-06 15:51:41vinay.sajipsetmessages: + msg373133
2020-07-06 14:59:19godot_gildorsetmessages: + msg373127
2020-07-06 14:29:02vinay.sajipsetmessages: + msg373120
2020-07-04 21:32:27serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg372999
2020-07-04 21:16:16vinay.sajipsetmessages: + msg372996
2020-07-02 20:54:35mivadesetnosy: + mivade
messages: + msg372883
2020-07-01 14:57:59godot_gildorsetversions: - Python 3.10
2020-07-01 06:05:30vinay.sajipsetnosy: + vinay.sajip

messages: + msg372731
versions: + Python 3.10, - Python 3.7
2020-06-30 23:40:09python-devsetkeywords: + patch
nosy: + python-dev

pull_requests: + pull_request20398
stage: patch review
2020-06-30 23:10:03godot_gildorcreate