This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python 3.7+ break on singledispatch_function.register(pseudo_type), which Python 3.6 accepted
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: AlexWaygood, Dutcho, enedil, gvanrossum, iritkatriel, levkivskyi, lukasz.langa, rhettinger, selik, serhiy.storchaka, techdragon, wrobell
Priority: normal Keywords: patch

Created on 2018-08-25 12:19 by Dutcho, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29508 closed AlexWaygood, 2021-11-09 22:39
Messages (13)
msg324061 - (view) Author: (Dutcho) Date: 2018-08-25 12:19
In Python 3.6, the argument to the register() method of a single-dispatch function (i.e. a function decorated by @functools.singledispatch) can be a 'type-like' class (pseudo-type?) from e.g. the typing module that supports isinstance().

The below demonstrates it works in Python 3.6:

$ py -3.6
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 03:37:03) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from functools import singledispatch
>>> from typing import Sequence
>>> isinstance([1,2,3], Sequence) # 'pseudo-type' okay with isinstance
True
>>> @singledispatch
... def f(arg):
...     print('unqualified', arg)
...
>>> @f.register(Sequence) # 'pseudo-type' okay with register
... def _(arg):
...     print('sequence', *arg)
...
>>> f(1)
unqualified 1
>>> f([1,2,3])
sequence 1 2 3

This code breaks in Python 3.7:

$ py
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from functools import singledispatch
>>> from typing import Sequence
>>> isinstance([1,2,3], Sequence) # 'pseudo-type' okay with isinstance
True
>>> @singledispatch
... def f(arg):
...     print('unqualified', arg)
...
>>> @f.register(Sequence)  # 'pseudo-type' NOT okay with register
... def _(arg):
...     print('sequence', *arg)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\Program Files\Python\Python37\lib\functools.py", line 801, in register
    f"Invalid first argument to `register()`: {cls!r}. "
TypeError: Invalid first argument to `register()`: typing.Sequence. Use either `@register(some_class)` or plain `@register` on an annotated function.

While agreeing a check on the register() arg IS required, the current check isinstance(cls, type) seems overly restrictive. This is especially true when considering the (welcome!) Python 3.7 use of annotations for single dispatch functions.
msg324081 - (view) Author: Michał Radwański (enedil) * Date: 2018-08-25 16:52
The change is not only in the added `isinstance` check:

~ $ python3.6 -q
>>> import typing
>>> isinstance(typing.Sequence, type)
True
>>>
~ $ python3.7 -q
>>> import typing
>>> isinstance(typing.Sequence, type)
False
>>>
msg324082 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2018-08-25 17:30
This is why we have 4 months of betas :-)

On one hand making objects in `typing` module not classes was intentional, but on another hand this use case looks totally fine.

I would say we could update the check in `functools` to accept more things. I am however not sure what to allow here in addition to proper classes, maybe just anything that overrides `__subclasscheck__` and/or `__instancecheck__`? I could imagine there might be some other objects that implement custom instance and class checks beyond `typing` that are rejected by current checks in `functools`.

As a temporary workaround you can use `collections.abc.Sequence`, this type is aliased by `typing.Sequence`, in fact all instance checks etc. are relayed to the former.
msg324248 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-08-28 12:58
> I would say we could update the check in `functools` to accept more things.

Could we revert abstract types in `typing` to respond True to `isinstance(..., type)` again?  I don't want singledispatch, or any other library like it, to have to special-case typing.   Especially for *non-generic* variants, responding True to `isinstance()` seems reasonable to me.  We cannot dispatch by `Sequence[int]` and some such so it makes sense for *those* to keep failing going forward.

The current suggested workaround of using abc.Sequence instead will work.
msg324251 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2018-08-28 13:20
> Could we revert abstract types in `typing` to respond True to `isinstance(..., type)` again?

No, making them classes will cause significant performance penalty (I don't remember numbers now, but importing `typing` may be twice slower). Plus this will re-introduce some bugs and inconsistencies.

> I don't want singledispatch, or any other library like it, to have to special-case typing.

I didn't propose this. I proposed to allow all objects that pretend to be types but are not actually instances of `type`.
msg326965 - (view) Author: Sam Bishop (techdragon) Date: 2018-10-03 11:54
Would the enhancements to resolve this, by making singledispatch accept more things, also resolve the AssertionError from functools.singledispatch when passing it custom types, or should I raise this as a separate issue? 

------------------------------------------------
from typing import NewType, List
from functools import singledispatch

@singledispatch
def fun(arg, verbose=False):
    if verbose:
        print("Let me just say,", end=" ")
    print(arg)

MyType = NewType('MyType', List[int])

@fun.register
def _(arg: MyType , verbose=False):
    if verbose:
        print("Strength in numbers, eh?", end=" ")
    print(arg)

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-49-e549104faa9a> in <module>
      5 
      6 @fun.register
----> 7 def _(arg: MyType , verbose=False):
      8     if verbose:
      9         print("Strength in numbers, eh?", end=" ")

~/.pyenv/versions/3.7.0/lib/python3.7/functools.py in register(cls, func)
    809             argname, cls = next(iter(get_type_hints(func).items()))
    810             assert isinstance(cls, type), (
--> 811                 f"Invalid annotation for {argname!r}. {cls!r} is not a class."
    812             )
    813         registry[cls] = func

AssertionError: Invalid annotation for 'arg'. <function NewType.<locals>.new_type at 0x10fcd7730> is not a class.
msg339071 - (view) Author: Michael Selik (selik) * Date: 2019-03-28 18:37
+1 for this use case. Until it's resolved, perhaps there should be a note in the singledispatch docs that types from the ``typing`` module should not be used?
msg401274 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-09-07 13:48
Reproduced on 3.11.

I've changed type because crash typically refers to segfault rather than an exception being raised.
msg409275 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-12-29 00:50
There's no point in lamenting the compatibility with Python 3.6, it's water under the bridge. Dispatching on types like list[int] or types generated by NewType is not realistic. Maybe the only thing left to do is to raise an error on registration when the type is e.g. list[int]? Currently that passes, but then attempting to dispatch on *anything* fails with TypeError: issubclass() argument 2 cannot be a parameterized generic
msg409276 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2021-12-29 01:01
Guido: Serhiy fixed this very recently in Issue46032.

The documentation should probably be improved, however, in my opinion; there's currently nothing officially stating that GenericAlias/NewType/typing aliases are unsupported.

Support for singledispatch.register(union_type) was also recently added in Issue46014, but has yet to be documented.
msg409296 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-12-29 08:29
The original issue is about dispatching on non-parametrized generics like typing.Sequence. isinstance([], typing.Sequence) works, so it could be possible to support dispatching on typing.Sequence. But I have doubts that it is worth to revive such feature, because it needs some effort, and you can use collections.abc.Sequence instead.

But there is a conflict between using annotations in singledispatch() and MyPy (issue46191).
msg409311 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-12-29 17:31
Let's close this issue as "won't fix" then. The solution "use collections.abc.Sequence" makes sense.
msg409358 - (view) Author: wrobell (wrobell) Date: 2021-12-30 10:44
To add bit more context, there is a discussion about Python types and
annotation types, where suggestion of closing this bug has been raised

    https://github.com/python/mypy/issues/9773

The discussion is about the need of distinction between Python types and
annotation types, and explicit terminology in the documentation.

Closing this bug gives clearer indication to tools like Mypy, how to deal
with annotation types used with singledispatch.

IMHO, it is small step towards more explict Python type system, whatever
its limitations.
History
Date User Action Args
2022-04-11 14:59:05adminsetgithub: 78679
2021-12-30 10:44:14wrobellsetnosy: + wrobell
messages: + msg409358
2021-12-30 08:13:16serhiy.storchakasetstatus: open -> closed
resolution: wont fix
stage: patch review -> resolved
2021-12-29 17:31:01gvanrossumsetmessages: + msg409311
2021-12-29 08:29:23serhiy.storchakasetmessages: + msg409296
2021-12-29 01:01:54AlexWaygoodsetmessages: + msg409276
2021-12-29 00:50:49gvanrossumsetnosy: + serhiy.storchaka, gvanrossum
messages: + msg409275
2021-11-09 22:39:55AlexWaygoodsetkeywords: + patch
nosy: + AlexWaygood

pull_requests: + pull_request27759
stage: patch review
2021-09-07 13:48:55iritkatrielsettitle: Python 3.7 breaks on singledispatch_function.register(pseudo_type), which Python 3.6 accepted -> Python 3.7+ break on singledispatch_function.register(pseudo_type), which Python 3.6 accepted
2021-09-07 13:48:23iritkatrielsetversions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.7
nosy: + iritkatriel

messages: + msg401274

type: crash -> behavior
2019-03-28 18:37:02seliksetnosy: + selik
messages: + msg339071
2018-10-03 15:18:56gvanrossumsetnosy: - gvanrossum
2018-10-03 11:54:35techdragonsetnosy: + techdragon
messages: + msg326965
2018-08-28 13:20:27levkivskyisetmessages: + msg324251
2018-08-28 12:58:02lukasz.langasetmessages: + msg324248
2018-08-27 19:50:50brett.cannonsetnosy: + lukasz.langa
2018-08-25 17:30:57levkivskyisetnosy: + rhettinger
messages: + msg324082
2018-08-25 17:14:04serhiy.storchakasetnosy: + gvanrossum, levkivskyi
2018-08-25 16:52:30enedilsetnosy: + enedil
messages: + msg324081
2018-08-25 12:19:29Dutchocreate