classification
Title: len() and iter() of ChainMap don't work with unhashable keys
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: ebw, r.david.murray, rhettinger, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-06-17 12:31 by serhiy.storchaka, last changed 2018-11-28 16:00 by ebw. This issue is now closed.

Messages (4)
msg296236 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-17 12:31
As documented ChainMap supports not just dicts, but mappings. A mapping can be implemented not only as a hash table, and mapping keys can be not hashable. But ChainMap.__len__ and ChainMap.__iter__ work only with hashable keys.

This may be considered just as a documentation issue. But I think it would be nice to add the support of unhashable keys in 3.7.
msg296257 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-06-18 04:10
No thank you.  It is perfectly reasonable to use set operations to make the implementation simple, fast, reliable, and suitable for most use cases.  Marking this as a rejected feature request.

As for documentation, I don't see any need to accommodate a pedantic reading of "mapping" in the general sense versus collections.Mapping in the formal sense.  Additional text would likely create more confusion than it solves and would likely get in the way of normal users who are just trying to learn how to use ChainMap.

If you truly feel that users need to know about every place that the  adjective "mapping" carries with it a dependency on hashability, then consider a FAQ entry or blog post on the subject.  Otherwise, we going have to replace term "mapping" with the more awkward "dict-like object".

Serhiy, while I admire your devotion to studying all the code in the standard library, it really isn't helpful to have you challenge and second guess every single design decision.  While you may have a ton of spare time to generate enormous numbers of PRs, feature requests, and bug reports, it is eating-up *all* of my development time.  It is a constant distraction from real work than needs to be done.   In particular, most of the issues seem to be "invented problems" that aren't based on any actual user need or reported problem.  Further, it risks excessive code churn, feature creep, risk of introducing new problems/incompatibilities, and destabilizes published code.  It may be fun to rewrite everything you look at, but it isn't a good practice.
msg296386 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-06-19 23:05
IMO, it would actually be surprising for ChainMap to support unhashable keys.  In Python, as Raymond indicates, "mapping" is pretty much synonymous with 'dict-like', and it would be where that was *not* true that we might add extra documentation.
msg330606 - (view) Author: ebw (ebw) Date: 2018-11-28 16:00
I actually have a real life use case of that problem. Using the eval() function of pandas need to use a custom resolver:

class Resolver(object):
    def __getitem__(self, name):
        if name == "test":
            return 0.5
        else:
            raise KeyError

resolvers = (Resolver(), )
pd.eval("test", resolvers=resolvers)

But pandas store the resolvers used for eval() in a DeepChainMap and uses bool(len(self.resolvers)) in the property has_resolvers. Which with my custom Resolver raise a KeyError because __getiem__ gets call with name = 0.

The way to solve/bypass the problem was to make Resolver a inherit from UserDict.

class Resolver(UserDict):
    def __init__(self, data={}):
        super(Resolver, self).__init__(data)
        self.data["test"] = None
    
    def __getitem__(self, name):
        if (name == "test"):
            return 0.5
        else:
            super(Resolver, self).__getitem__(name)

resolvers = (Resolver(), )
pd.eval("test", resolvers=resolvers)

I had to add a dummy "test" value to Resolver.data dict to avoid len(self.resolvers) to be 0. I tried to implement Resolver.__len__, keys... but it didn't help has ChainMap compute the len using set().union(*self.maps).
History
Date User Action Args
2018-11-28 16:00:26ebwsetnosy: + ebw
messages: + msg330606
2017-06-19 23:05:10r.david.murraysetnosy: + r.david.murray
messages: + msg296386
2017-06-18 04:10:55rhettingersetstatus: open -> closed
versions: - Python 2.7, Python 3.5, Python 3.6
type: behavior -> enhancement
messages: + msg296257

resolution: rejected
stage: resolved
2017-06-17 12:31:43serhiy.storchakacreate