classification
Title: Loosening | and |= operator type checking restriction
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: brandtbucher, mrabarnett, rhettinger, steven.daprano, uriyyo, veky
Priority: normal Keywords: patch

Created on 2021-01-31 16:14 by Siy_V, last changed 2021-02-01 18:07 by brandtbucher.

Pull Requests
URL Status Linked Edit
PR 24413 closed uriyyo, 2021-02-01 14:45
Messages (7)
msg386028 - (view) Author: Siyavash (Siy_V) Date: 2021-01-31 16:14
| and |= operators work only if the operands are of 'dict' type. This means that they can't be used with some of the user defined mapping objects.

The case that especially caught my attention was 'SignalDict' type in decimal standard library. While 'SignalDict' has got an update method like 'dict', it can not be updated with |= operator. The following code results in a TypeError:

decimal.getcontext().traps |= {decimal.DivisionByZero:False}
msg386030 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-01-31 17:48
The operators | and |= always call __or__() and __ior__().  Support for these methods is optional.  Some mapping classes support both methods, some only support __or__, and some support neither one.

Adding support is done is done on a case by case basis.  SignalDict could be updated if needed.

To make it easier for user defined classes, two new collections.ABCs could be added, MappingWithOr and MutableMappingWithIor.  That is similar to what we had done in Python 2.7 with UserDict and IterableUserDict.
msg386066 - (view) Author: Yurii Karabas (uriyyo) * (Python triager) Date: 2021-02-01 13:19
Hi Raymond,

I was trying to add __or__ method for collections.abc.Mapping, __ior__ method for collections.abc.MutableMapping, link to issue - https://bugs.python.org/issue42841.

I can extract code and move it to separate classes as you propose above.
What do you think about that?
msg386068 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-02-01 13:34
Because ABCs set minimum standards for compliance, they can't be extended without breaking existing code that registers to the ABC.

Brandt added __or/ior__ support to some of the mapping classes in the standard library, but it is a fact of life that there will always be some mapping classes that don't have __or/ior__ support.  It is an optional feature.

It is possible add new ABC subclasses with the extra methods, but I don't think that is worth it.  The effort for a mapping class maintainer to change which class it inherits from or registers to is about equal to the effort to just add the two methods.

So, I think your best bet is to propose PRs for individual mapping classes on a case-by-case basis.
msg386083 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-02-01 17:28
Yurii, I agree with Raymond that it's probably not worth adding new ABCs for this.

I'm happy to consider patches that add these operators to concrete stdlib mappings on a case-by-case basis. However, please be aware that we already went through a fairly comprehensive review of each dict/Mapping/MutableMapping subclass (real or virtual) in the entire stdlib to see if they would benefit here. I don't think we bothered updating *any* classes that weren't publicly documented, which is probably why "SignalDict" was left untouched.

It's also important to remember that the motivating use-case for PEP 584 was binary "|", and that "|=" just logically followed from that change. In this specific case, I think just using the good ol' update method (or even item assignment) is probably good enough.

I'm not very familiar with how SignalDict is used, though: do we ever actually want to create a new updated copy using binary "|"? That would probably be enough to convince me.
msg386084 - (view) Author: Vedran Čačić (veky) * Date: 2021-02-01 17:38
@Raymond: why can't ABCs have _default_ implementation of | and |=, in terms of other supported methods (such as update)? It seems to me this would be the best of both worlds.
msg386086 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-02-01 18:07
Any class that registers itself as a Mapping/MutableMapping using their "register" methods (*not* through inheritance) will be lacking the new methods. This includes all C extensions.

Further, binary "|" would require subclasses of these ABCs to have a concrete way of creating a new instance (we don't have anything like Set._from_iterable for this). One option could be to just call self.__class__({**self, **other}), but a cursory scan of the stdlib shows that many classes don't have compatible signatures, and the we would get lots of surprising/broken behavior if we just suddenly introduced this.
History
Date User Action Args
2021-02-01 18:07:13brandtbuchersetmessages: + msg386086
2021-02-01 17:38:40vekysetnosy: + veky
messages: + msg386084
2021-02-01 17:28:35brandtbuchersetmessages: + msg386083
components: + Library (Lib), - Regular Expressions
versions: + Python 3.10, - Python 3.9
2021-02-01 14:45:59uriyyosetkeywords: + patch
stage: patch review
pull_requests: + pull_request23228
2021-02-01 13:34:01rhettingersetmessages: + msg386068
2021-02-01 13:19:10uriyyosetnosy: + uriyyo
messages: + msg386066
2021-01-31 17:48:19rhettingersetnosy: + rhettinger
messages: + msg386030
2021-01-31 16:15:39Siy_Vsetnosy: - ezio.melotti, Siy_V
2021-01-31 16:14:39Siy_Vcreate