classification
Title: Discrepancy between isinstance() and issubclass() for union types
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, kj, miss-islington, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2021-07-12 05:33 by serhiy.storchaka, last changed 2021-07-14 05:21 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27120 merged serhiy.storchaka, 2021-07-13 17:29
PR 27132 merged miss-islington, 2021-07-14 04:35
Messages (11)
msg397281 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-12 05:33
1. Different handling of None:

>>> isinstance(None, int | type(None))
True
>>> issubclass(type(None), int | type(None))
True
>>> isinstance(None, int | None)
True
>>> issubclass(type(None), int | None)
False

2. Different handling of virtual subclasses:

>>> import collections.abc
>>> isinstance({}, int | collections.abc.Mapping)
True
>>> issubclass(dict, int | collections.abc.Mapping)
False

I do not know what behavior is correct.
msg397355 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-07-12 20:12
Ken Jin, should we treat type(None) as a subclass of int|None?
msg397385 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-13 06:19
I have found this when refactored the code of Objects/unionobject.c. So I have a patch which fixes this, but I want to make sure what behavior is considered correct.
msg397389 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-07-13 09:29
@Serhiy
> 2. Different handling of virtual subclasses:

This looks like a bug. I think the union form of isinstance/issubclass should have the same behavior as the tuple form. We promise checking of virtual classes in the docs for union types:
https://docs.python.org/3.10/library/functions.html#issubclass

The fix seems to be changing one line of code in Objects/unionobject.c from PyType_IsSubtype to PyObject_IsSubclass. Since it isn't a large change and it's a bug, I think we can backport it to 3.10 too.

@Serhiy and Guido,
> 1. Different handling of None:

> Ken Jin, should we treat type(None) as a subclass of int|None?

I think we can avoid this special case altogether by converting all None to type(None) when creating _Py_Union. This is what the typing.Union counterpart does, and it is also what PEP 484 says about None https://www.python.org/dev/peps/pep-0484/#using-none:

>>> Union[None, str].__args__
(<class 'NoneType'>, <class 'str'>)

Though I know some people are opinionated about None -> type(None).

This would require adding one more check at https://github.com/python/cpython/blob/3.10/Objects/unionobject.c#L266 .
Maybe this?

if(Py_IsNone(i_element))
    i_element = &_PyNone_Type
msg397405 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-07-13 14:22
Converting None to type(None) is fine, as long as the str() /repr()
converts it back, e.g. repr(int | None) should print just that, not
NoneType.--
--Guido (mobile)
msg397417 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-07-13 15:16
@Serhiy, can I work on converting None to type(None) please?
msg397430 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-13 17:16
3. There is also a crash in isinstance():

>>> class BadMeta(type):
...     def __instancecheck__(cls, inst):
...         1/0
... 
>>> x = int | BadMeta('A', (), {})
>>> isinstance([], x)
Fatal Python error: _Py_CheckFunctionResult: a function returned a result with an exception set
Python runtime state: initialized
Traceback (most recent call last):
  File "<stdin>", line 3, in __instancecheck__
ZeroDivisionError: division by zero

The above exception was the direct cause of the following exception:

SystemError: <built-in method __instancecheck__ of types.Union object at 0x7f024bbb8960> returned a result with an exception set

Current thread 0x00007f024beb1280 (most recent call first):
  File "<stdin>", line 1 in <module>
Aborted (core dumped)
msg397431 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-13 17:31
PR 27120 fixes __instancecheck__ and __subclasscheck__. Converting None to type(None) will be done in other PR.
msg397432 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-07-13 17:34
> 3. There is also a crash in isinstance():

That's unfortunate :(.

Serhiy, thanks for catching all these bugs in union. I recently realized you probably made 50% of all bug reports for union and they're very much appreciated :).

> Converting None to type(None) will be done in other PR.

Alright. Do tell me if you're too busy and want me to take it up instead. I'll be glad to help.
msg397463 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-14 04:35
New changeset 81989058de381108dfd0a4255b93d4fb34417002 by Serhiy Storchaka in branch 'main':
bpo-44606: Fix __instancecheck__ and __subclasscheck__ for the union type. (GH-27120)
https://github.com/python/cpython/commit/81989058de381108dfd0a4255b93d4fb34417002
msg397464 - (view) Author: miss-islington (miss-islington) Date: 2021-07-14 04:55
New changeset b42eee78e7651693aa38c390f577e5d499dcf55d by Miss Islington (bot) in branch '3.10':
bpo-44606: Fix __instancecheck__ and __subclasscheck__ for the union type. (GH-27120)
https://github.com/python/cpython/commit/b42eee78e7651693aa38c390f577e5d499dcf55d
History
Date User Action Args
2021-07-14 05:21:16serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-07-14 04:55:49miss-islingtonsetmessages: + msg397464
2021-07-14 04:35:50miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25675
2021-07-14 04:35:48serhiy.storchakasetmessages: + msg397463
2021-07-13 17:34:37kjsetmessages: + msg397432
2021-07-13 17:31:25serhiy.storchakasetmessages: + msg397431
2021-07-13 17:30:02serhiy.storchakasettype: behavior -> crash
2021-07-13 17:29:43serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request25665
2021-07-13 17:16:19serhiy.storchakasetmessages: + msg397430
2021-07-13 15:16:45kjsetmessages: + msg397417
2021-07-13 14:22:41gvanrossumsetmessages: + msg397405
2021-07-13 09:29:29kjsetmessages: + msg397389
2021-07-13 06:19:02serhiy.storchakasetmessages: + msg397385
2021-07-12 20:12:59gvanrossumsetnosy: + kj
messages: + msg397355
2021-07-12 05:33:29serhiy.storchakacreate