Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

functools.singledispatch does not support Union types #90172

Closed
evansd mannequin opened this issue Dec 8, 2021 · 9 comments
Closed

functools.singledispatch does not support Union types #90172

evansd mannequin opened this issue Dec 8, 2021 · 9 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@evansd
Copy link
Mannequin

evansd mannequin commented Dec 8, 2021

BPO 46014
Nosy @rhettinger, @ambv, @uriyyo, @AlexWaygood, @evansd
PRs
  • bpo-46014: Add ability to use typing.Union with singledispatch #30017
  • bpo-46014: Add docs regarding functools.singledispatch changes #32282
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-12-10.23:29:02.779>
    created_at = <Date 2021-12-08.13:34:12.625>
    labels = ['type-feature', 'library', '3.9', '3.10', '3.11']
    title = 'functools.singledispatch does not support Union types'
    updated_at = <Date 2022-04-03.08:22:50.956>
    user = 'https://github.com/evansd'

    bugs.python.org fields:

    activity = <Date 2022-04-03.08:22:50.956>
    actor = 'uriyyo'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-12-10.23:29:02.779>
    closer = 'lukasz.langa'
    components = ['Library (Lib)']
    creation = <Date 2021-12-08.13:34:12.625>
    creator = 'evansd'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 46014
    keywords = ['patch']
    message_count = 6.0
    messages = ['408018', '408261', '408262', '408272', '409366', '409369']
    nosy_count = 6.0
    nosy_names = ['rhettinger', 'wrobell', 'lukasz.langa', 'uriyyo', 'AlexWaygood', 'evansd']
    pr_nums = ['30017', '32282']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue46014'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    @evansd
    Copy link
    Mannequin Author

    evansd mannequin commented Dec 8, 2021

    It's not currently possible to use singledispatch with a function annotated with a Union type, although the desired behaviour is clear.

    Example:

        from functools import singledispatch
        from typing import Union
    
        @singledispatch
        def test(arg):
            raise ValueError(arg)
    
        @test.register
        def _(arg: Union[int, float]):
            print(f"{arg} is a number")
    
        test(1)
    

    This dies with:

    TypeError: Invalid annotation for 'arg'. typing.Union[int, float] is not a class.
    

    I've implemented a workaround which digs the types out of the union and registers them one-by-one:

        from functools import singledispatch
        from typing import Union, get_type_hints
    
        def register_for_union_type(target_fn):
            def decorator(fn):
                arg_type = list(get_type_hints(fn).values())[0]
                assert arg_type.__origin__ is Union
                for type_ in arg_type.__args__:
                    fn = target_fn.register(type_)(fn)
                return fn
    
            return decorator
    
        @singledispatch
        def test(arg):
            raise ValueError(arg)
    
        @register_for_union_type(test)
        def _(arg: Union[int, float]):
            print(f"{arg} is a number")
    
        test(1)
    
    

    Looking at the source for singledispatch is doesn't look like it would too difficult to add support for something like this, though of course there may be subtleties I've missed. Would you be open to a patch?

    The only other mentions I've found of this online are:
    https://mail.python.org/archives/list/python-ideas@python.org/thread/HF5HDXQ2SXZHO3TWODIRQATTE4TCAWPL/
    https://stackoverflow.com/questions/61721761/type-hints-and-singledispatch-how-do-i-include-union-in-an-extensible-w

    @evansd evansd mannequin added 3.7 (EOL) end of life 3.8 only security fixes 3.10 only security fixes 3.11 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Dec 8, 2021
    @AlexWaygood AlexWaygood removed 3.7 (EOL) end of life 3.8 only security fixes labels Dec 8, 2021
    @ambv
    Copy link
    Contributor

    ambv commented Dec 10, 2021

    New changeset 3cb357a by Yurii Karabas in branch 'main':
    bpo-46014: Add ability to use typing.Union with singledispatch (GH-30017)
    3cb357a

    @ambv
    Copy link
    Contributor

    ambv commented Dec 10, 2021

    Support added as an enhancement in Python 3.11. Thanks, Yurii! ✨ 🍰 ✨

    @ambv ambv closed this as completed Dec 10, 2021
    @ambv ambv closed this as completed Dec 10, 2021
    @AlexWaygood
    Copy link
    Member

    This is awesome! Should a note be added to the functools documentation mentioning the new feature? (Genuine question — I'm not sure whether it's necessary myself.)

    @wrobell
    Copy link
    Mannequin

    wrobell mannequin commented Dec 30, 2021

    Will it support Optional as well?

    According to PEP-604, these two shall be equivalent (using Python 3.10 below)?

    type(int | None)
    types.UnionType
    
    type(tp.Optional[int])
    typing._UnionGenericAlias
    

    Also, IMHO, the documentation of singledispatch should be improved. It needs to state that only Python types are supported with few exceptions (union type now), but other annotation types are not supported, i.e. generics like list[int].

    @AlexWaygood
    Copy link
    Member

    >>> from typing import Optional, get_origin
    >>> get_origin(Optional[int])
    typing.Union
    

    ^Because of that, it will work with typing.Optional as well as typing.Union and types.UnionType, yes.

    I am planning on submitting a docs PR at some point in the next few days (probably linked to a new BPO issue).

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @iritkatriel iritkatriel reopened this Apr 11, 2022
    @iritkatriel
    Copy link
    Member

    Reopening as per #32282 (comment)

    @JelleZijlstra
    Copy link
    Member

    >>> from typing import Optional, get_origin
    >>> get_origin(Optional[int])
    typing.Union
    

    ^Because of that, it will work with typing.Optional as well as typing.Union and types.UnionType, yes.

    It does indeed work (continuing from the OP's example):

    >>> @test.register
    ... def _(arg: str  | None):
    ...     print(f"{arg} may be a string")
    ... 
    >>> test(None)
    None may be a string
    >>> test("x")
    x may be a string

    But it's not obvious that it should, because None isn't a type. It apparently works because None gets turned into type(None) somewhere along the way in typing.py, but this seems worth testing explicitly in test_functools.

    So anyone up for adding a test case that uses an Optional type?

    JelleZijlstra added a commit that referenced this issue May 3, 2022
    …one type (#92174)
    
    Signed-off-by: prwatson <prwatson@redhat.com>
    
    Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
    @JelleZijlstra
    Copy link
    Member

    Thanks @Thaddeus1499 for contributing more tests.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants