classification
Title: functools: singledispatchmethod doesn't work with classmethod
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: AlexWaygood, EugenePY, FFY00, Viktor Roytman, dmkulazhenko, glyph, levkivskyi, lukasz.langa, markgrandi, mental, miss-islington, ncoghlan, xtreak
Priority: normal Keywords: patch

Created on 2020-02-18 19:16 by Viktor Roytman, last changed 2021-10-28 16:03 by lukasz.langa. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29034 merged AlexWaygood, 2021-10-18 15:49
PR 29072 merged miss-islington, 2021-10-19 20:30
PR 29087 merged AlexWaygood, 2021-10-20 09:33
Messages (15)
msg362233 - (view) Author: Viktor Roytman (Viktor Roytman) * Date: 2020-02-18 19:16
I couldn't get the example given for the interaction between @singledispatchmethod and @classmethod to work https://docs.python.org/3/library/functools.html?highlight=singledispatch#functools.singledispatchmethod

    from functools import singledispatchmethod
    
    
    class Negator:
        @singledispatchmethod
        @classmethod
        def neg(cls, arg):
            raise NotImplementedError("Cannot negate a")
    
        @neg.register
        @classmethod
        def _(cls, arg: int):
            return -arg
    
        @neg.register
        @classmethod
        def _(cls, arg: bool):
            return not arg
    
    
    if __name__ == "__main__":
        print(Negator.neg(0))
        print(Negator.neg(False))

Leads to

    $ python -m bad_classmethod_as_documented
    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/home/viktor/scratch/bad_classmethod_as_documented.py", line 4, in <module>
        class Negator:
      File "/home/viktor/scratch/bad_classmethod_as_documented.py", line 12, in Negator
        def _(cls, arg: int):
      File "/usr/lib/python3.8/functools.py", line 906, in register
        return self.dispatcher.register(cls, func=method)
      File "/usr/lib/python3.8/functools.py", line 848, in register
        raise TypeError(
    TypeError: Invalid first argument to `register()`: <classmethod object at 0x7f37d1469070>. Use either `@register(some_class)` or plain `@register` on an annotated function.

Curiously, @staticmethod does work, but not as documented (don't decorate the actual implementations):

    from functools import singledispatchmethod
    
    
    class Negator:
        @singledispatchmethod
        @staticmethod
        def neg(arg):
            raise NotImplementedError("Cannot negate a")
    
        @neg.register
        def _(arg: int):
            return -arg
    
        @neg.register
        def _(arg: bool):
            return not arg
    
    
    if __name__ == "__main__":
        print(Negator.neg(0))
        print(Negator.neg(False))

Leads to

    $ python -m good_staticmethod
    0
    True

Removing @classmethod from the implementation methods doesn't work, though

    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/home/viktor/scratch/bad_classmethod_alternative.py", line 20, in <module>
        print(Negator.neg(0))
      File "/usr/lib/python3.8/functools.py", line 911, in _method
        return method.__get__(obj, cls)(*args, **kwargs)
    TypeError: _() missing 1 required positional argument: 'arg'
msg362253 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2020-02-19 05:36
I guess the method checks for annotation on cls [0] which will be classmethod/staticmethod object in the report and won't have annotations. The annotations should be looked up in the function the classmethod/staticmethod decorator wraps around as in cls.__func__ . Something like below so that the right annotations are picked up. In addition to this the registry should store the type annotation as key to cls or cls.__func__ depending on normal method or classmethod/staticmethod.

diff --git Lib/functools.py Lib/functools.py
index 050bec8605..a66711208d 100644
--- Lib/functools.py
+++ Lib/functools.py
@@ -1073,24 +1073,33 @@ def singledispatch(func):
         if func is None:
             if isinstance(cls, type):
                 return lambda f: register(cls, f)
-            ann = getattr(cls, '__annotations__', {})
+            if isinstance(cls, (classmethod, staticmethod)):
+                ann = getattr(cls.__func__, '__annotations__', {})
+                func = cls.__func__
+            else:
+                ann = getattr(cls, '__annotations__', {})
+                func = cls

[0] https://github.com/python/cpython/blob/ab6423fe2de0ed5f8a0dc86a9c7070229326b0f0/Lib/functools.py#L1076
msg362434 - (view) Author: Viktor Roytman (Viktor Roytman) * Date: 2020-02-21 21:21
I tried to apply this change but it didn't work, failing with this error

    $ ~/.pyenv/versions/3.8.1/bin/python -m bad_classmethod_as_documented
    Traceback (most recent call last):
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 193, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/home/viktor/scratch/bad_classmethod_as_documented.py", line 4, in <module>
        class Negator:
      File "/home/viktor/scratch/bad_classmethod_as_documented.py", line 12, in Negator
        def _(cls, arg: int):
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/functools.py", line 1006, in register
        return self.dispatcher.register(cls, func=method)
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/functools.py", line 959, in register
        argname, cls = next(iter(get_type_hints(func).items()))
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/typing.py", line 1252, in get_type_hints
        raise TypeError('{!r} is not a module, class, method, '
    TypeError: <classmethod object at 0x7f84e1e69c40> is not a module, class, method, or function.

After digging around a bit, this diff seems to work (not sure if there's a better way to do it) (also this one doesn't seem to care whether @staticmethod is applied to the implementation methods):

$ diff -u functools-orig.py functools.py
--- functools-orig.py	2020-02-21 16:14:56.141934001 -0500
+++ functools.py	2020-02-21 16:17:19.959905849 -0500
@@ -843,14 +843,18 @@
         if func is None:
             if isinstance(cls, type):
                 return lambda f: register(cls, f)
-            ann = getattr(cls, '__annotations__', {})
+            if isinstance(cls, (classmethod, staticmethod)):
+                ann = getattr(cls.__func__, '__annotations__', {})
+                func = cls.__func__
+            else:
+                ann = getattr(cls, '__annotations__', {})
+                func = cls
             if not ann:
                 raise TypeError(
                     f"Invalid first argument to `register()`: {cls!r}. "
                     f"Use either `@register(some_class)` or plain `@register` "
                     f"on an annotated function."
                 )
-            func = cls
 
             # only import typing if annotation parsing is necessary
             from typing import get_type_hints
@@ -908,6 +912,8 @@
     def __get__(self, obj, cls=None):
         def _method(*args, **kwargs):
             method = self.dispatcher.dispatch(args[0].__class__)
+            if isinstance(self.func, classmethod):
+                return method.__get__(obj, cls)(cls, *args, **kwargs)
             return method.__get__(obj, cls)(*args, **kwargs)
 
         _method.__isabstractmethod__ = self.__isabstractmethod__
msg362442 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2020-02-22 01:16
Sorry, I had the part only to detect annotations attached. My part is something similar to yours except it stores the appropriate function in the registry itself instead of passing the arguments at __get__ .
msg371230 - (view) Author: Viktor Roytman (Viktor Roytman) * Date: 2020-06-10 18:13
Sorry to bump this after months of inactivity but is there something I need to do here?
msg372405 - (view) Author: Mark Grandi (markgrandi) * Date: 2020-06-26 03:03
same issue here, if we can't fix it then maybe we should edit the documentation to not suggest that @classmethod works?
msg378344 - (view) Author: Glyph Lefkowitz (glyph) (Python triager) Date: 2020-10-09 20:00
I also just discovered this.  I thought it must have regressed at some point but I see the docs say "new in 3.8" and I'm using 3.8.

Is there a "no CI for examples in the docs" issue that this could be linked to?
msg380221 - (view) Author: Eugene-Yuan Kou (EugenePY) Date: 2020-11-02 14:00
Hi, I also encounter to the problem, and I have give my attempt to make the singledispatchmethod support the classmethod, and staticmethod with type annotation. I also adding two tests. Please refer to my fork.  I will trying to make a pull request

https://github.com/EugenePY/cpython/compare/3.8...fix-issue-39679
msg395910 - (view) Author: Dmitry Kulazhenko (dmkulazhenko) Date: 2021-06-16 07:50
Based on what I've read, workaround:


from functools import singledispatchmethod


def _register(self, cls, method=None):
    if hasattr(cls, "__func__"):
        setattr(cls, "__annotations__", cls.__func__.__annotations__)
    return self.dispatcher.register(cls, func=method)


singledispatchmethod.register = _register
msg404147 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2021-10-17 21:40
Happily, this bug appears to have been resolved in Python 3.10 due to the fact that a `classmethod` wrapping a function `F` will now have an `__annotations__` dict equal to `F`.

In Python 3.9:

```
>>> x = lambda y: y
>>> x.__annotations__ = {'y': int}
>>> c = classmethod(x)
>>> c.__annotations__
Traceback (most recent call last):
  File "<pyshell#37>", line 1, in <module>
    c.__annotations__
AttributeError: 'classmethod' object has no attribute '__annotations__'
```

In Python 3.10:

```
x = lambda y: y
x.__annotations__ = {'y': int}
c = classmethod(x)
c.__annotations__
{'y': <class 'int'>}
```

This change appears to have resolved the bug in `functools.singledispatchmethod`. The bug remains in Python 3.9, however.
msg404348 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-10-19 20:30
New changeset ad6d162e518963711d24c80f1b7d6079bd437584 by Alex Waygood in branch 'main':
bpo-39679: Add tests for classmethod/staticmethod singledispatchmethods (GH-29034)
https://github.com/python/cpython/commit/ad6d162e518963711d24c80f1b7d6079bd437584
msg404362 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-10-19 22:07
New changeset c15ba304f35362470e29ea5626fed28366bc9571 by Miss Islington (bot) in branch '3.10':
bpo-39679: Add tests for classmethod/staticmethod singledispatchmethods (GH-29034) (GH-29072)
https://github.com/python/cpython/commit/c15ba304f35362470e29ea5626fed28366bc9571
msg404363 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-10-19 22:08
Thanks for the new tests, Alex. Not closing this just yet because maybe we find a way to fix it in 3.9.8+.
msg405195 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-10-28 16:02
New changeset 97388c204b557f30e48a2b2ef826868702204cf2 by Alex Waygood in branch '3.9':
[3.9] bpo-39679: Fix `singledispatchmethod` `classmethod`/`staticmethod` bug (GH-29087)
https://github.com/python/cpython/commit/97388c204b557f30e48a2b2ef826868702204cf2
msg405196 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-10-28 16:03
Thanks for the 3.9 fix, Alex! ✨ 🍰 ✨
History
Date User Action Args
2021-10-28 16:03:07lukasz.langasetstatus: open -> closed
resolution: fixed
messages: + msg405196

stage: patch review -> resolved
2021-10-28 16:02:11lukasz.langasetmessages: + msg405195
2021-10-20 09:33:05AlexWaygoodsetpull_requests: + pull_request27353
2021-10-19 22:08:15lukasz.langasetmessages: + msg404363
2021-10-19 22:07:17lukasz.langasetmessages: + msg404362
2021-10-19 20:30:48miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27340
2021-10-19 20:30:33lukasz.langasetmessages: + msg404348
2021-10-18 15:49:19AlexWaygoodsetkeywords: + patch
stage: patch review
pull_requests: + pull_request27305
2021-10-17 21:40:22AlexWaygoodsetnosy: + AlexWaygood

messages: + msg404147
versions: - Python 3.8
2021-06-16 07:51:46dmkulazhenkosetnosy: + ncoghlan, glyph, lukasz.langa, markgrandi, levkivskyi, xtreak, mental, Viktor Roytman, FFY00, EugenePY
2021-06-16 07:50:42dmkulazhenkosetnosy: + dmkulazhenko, - ncoghlan, glyph, lukasz.langa, markgrandi, levkivskyi, xtreak, mental, Viktor Roytman, FFY00, EugenePY
messages: + msg395910
2020-11-10 06:28:14mentalsetnosy: + mental
2020-11-02 14:00:55EugenePYsetnosy: + EugenePY
messages: + msg380221
2020-10-12 12:50:25FFY00setnosy: + FFY00
2020-10-09 20:00:38glyphsetnosy: + glyph
messages: + msg378344
2020-06-26 03:03:48markgrandisetnosy: + markgrandi
messages: + msg372405
2020-06-10 18:13:24Viktor Roytmansetmessages: + msg371230
2020-02-22 01:16:11xtreaksetmessages: + msg362442
2020-02-21 21:21:43Viktor Roytmansetmessages: + msg362434
2020-02-19 05:36:16xtreaksetnosy: + xtreak, ncoghlan, levkivskyi
messages: + msg362253
2020-02-19 03:54:23xtreaksetnosy: + lukasz.langa
2020-02-18 19:16:15Viktor Roytmancreate