classification
Title: Overloading int.__pow__ does not work
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: zseil Nosy List: BreamoreBoy, ajaksu2, georg.brandl, mark.dickinson, rhettinger, serhiy.storchaka, terry.reedy, zseil
Priority: normal Keywords: patch

Created on 2007-04-04 23:44 by terry.reedy, last changed 2013-01-05 12:35 by mark.dickinson. This issue is now closed.

Files
File name Uploaded Description Edit
mixing_slots.diff zseil, 2007-04-05 01:47 patch against trunk revision 54690 review
slot_inheritance.diff zseil, 2008-11-17 12:22 An old patch with a few more fixes review
time_slot_inheritance.py zseil, 2008-11-17 12:26 The timeit script from the last message
Messages (10)
msg31702 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2007-04-04 23:44
From c.l.p: in response to glitch report, 2nd person showed that it is specific to __pow__

>>> class MyInt(int):
...     __sub__ = int.__add__
# similar lines for 9 other methods omitted
...     __or__ = int.__add__
...     __pow__ = int.__add__
...
>>> i = MyInt(42)
>>> i + 3
45
>>> i - 3
45
# similar outputs omitted
>>> i | 3
45
>>> i ** 3
74088

Another person (3rd) confirmed presence in 2.5 
Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] 
msg31703 - (view) Author: Ziga Seilnacht (zseil) * (Python committer) Date: 2007-04-05 01:47
Here is a patch (with tests) that should fix this.
There was another problem when the slot wrapper
came from a different type:

>>> class MyInt(int):
...     __mul__ = float.__add__
...
>>> MyInt(3) * 3
9

This now raises a TypeError.

File Added: mixing_slots.diff
msg31704 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2007-04-05 04:54
I'll take a look at your patch.

The root problem is that the wrapper functions will return NULL if they are fed the wrong number of arguments (i.e. the int.__add__ wrapper expects exactly two arguments but the ** call provides three) -- you will see a similar result if int.__neg__ or int.__invert__ are assigned to __add__.   It looks like an upstream step is seeing the NULL and deciding that it needs to look to skip the given method call and instead try the base the class.

Am lowering the priority because there is no good use case for deliberate argument mismatches in the override.  Am leaving the bug open because we want the appropriate error message to surface.

msg31705 - (view) Author: Ziga Seilnacht (zseil) * (Python committer) Date: 2007-04-06 23:27
Hi Raymond,

The problem is not in conversion functions, but in
slot inheritance.  If you run the example under a
debugger you can see that

MyInt->tp_as_number->nb_power == int->->tp_as_number->nb_power.

This happens because update_one_slot() doesn't fall
back to the generic slot function if a SlotWrapper in
type's dict has a different conversion function or if
it comes from an unrelated type.  The patch simply falls
back to the generic slot function in this case.
msg75934 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2008-11-16 12:25
Georg, do you want to take a look at this patch.  I've neglected it
because I don't see why the argument signature matters (__neg__ and
__pow__ take a different number of arguments than the other magic
numeric methods that work just fine).
msg75957 - (view) Author: Ziga Seilnacht (zseil) * (Python committer) Date: 2008-11-17 12:22
Hi Raymond,

The signature matters because the current code in update_one_slot()
forgets to set the use_generic flag when slots have different wrappers.
This causes that the slot from the base class is left in the new type.
Slots have different wrappers when their signature differs.

I also found an old patch for this issue that fixes a few more corner
cases, but I can't remember if it is finished and it doesn't apply to
current trunk. I'll try to fix that soon, but here is the original
message and patch:

"""
Here is a new version of the patch.  It fixes another bug and
restores the optimization that was lost when dict.__getitem__
and list.__getitem__ became methods with a METH_COEXIST flag.

The bug that this patch fixes shows when a type has the wrong
__new__ method:

>>> class C(object):
...     __new__ = int.__new__
...
>>> C()            # should raise a TypeError, but doesn't
>>> C.__new__(C)   # raises a TypeError, as expected

This means that Guido should buy me a beer :).

Adding __getitem__ to dict's and list's methods slowed down the
corresponding operation for subclasses that don't overwrite
__getitem__.  update_one_slot() installs a generic wrapper when
a __special__ method is not a WrapperDescriptor.  As a consequence,
this code is slower:

>>> class MyList(list):
...     pass
...
>>> L = MyList([0])
>>> L[0]           # currently slower than without METH_COEXIST

set.__contains__ and dict.__contains__ have the same problem.
I'll attach a timeit script that shows the difference.

The basic idea of this patch is that when all of the __special__
methods for a particular slot are inherited from the same base,
we can safely use that base's slot for the subtype.  The patch
compares __special__ methods by identity, which eliminates all
of the problems listed in this bug report.  For more details,
see the comments in the patch.

The patch contains only the tests for the bugs reported here;
the old behavior is already thoroughly tested in test_descr's
dynamics(), overloading(), subclasspropagation() and other
functions.

The patch introduces a slight incompatibility with Python 2.4 and
2.5; code that calls PySequence_GetItem(dict_subclass, index) for
dict subclasses can now fail, because tp_as_sequence->sq_item gets
filled only if the subclass overwrites the __getitem__ method.

Here are the results of running the script on a trunk build:

 list test: 0.585701534692
 dict test: 0.562311969817
  set test: 0.468992008761
class test: 0.235781083909

and results of running the script with this patch applied:

 list test: 0.167487487935
 dict test: 0.149341885631
  set test: 0.153591029027
class test: 0.211393347479
"""
msg86635 - (view) Author: Daniel Diniz (ajaksu2) (Python triager) Date: 2009-04-27 00:55
Confirmed in py3k and trunk.
msg110577 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-17 16:28
Could someone please review this patch to typeobject.c.
msg179061 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-04 18:12
This behavior doesn't reproduced more on 2.7 and 3.2+.

>>> class MyInt(int):
...     __pow__ = int.__add__
... 
>>> i = MyInt(42)
>>> i**3
45
msg179119 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-01-05 12:35
Yep, this was fixed as part of the issue 14658 fix.
History
Date User Action Args
2013-01-05 12:35:24mark.dickinsonsetmessages: + msg179119
2013-01-04 18:12:47serhiy.storchakasetstatus: open -> closed

nosy: + serhiy.storchaka
messages: + msg179061

resolution: out of date
stage: patch review -> resolved
2010-07-17 16:28:36BreamoreBoysetversions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 3.0
nosy: + BreamoreBoy

messages: + msg110577

components: + Extension Modules, - None
2009-04-27 00:55:15ajaksu2setversions: + Python 2.6, Python 3.0, - Python 2.5
nosy: + ajaksu2, mark.dickinson

messages: + msg86635

type: behavior
stage: patch review
2008-11-18 08:03:27georg.brandlsetassignee: georg.brandl -> zseil
2008-11-17 12:26:54zseilsetfiles: + time_slot_inheritance.py
2008-11-17 12:22:54zseilsetfiles: + slot_inheritance.diff
keywords: + patch
messages: + msg75957
2008-11-16 12:25:08rhettingersetpriority: low -> normal
assignee: rhettinger -> georg.brandl
messages: + msg75934
nosy: + georg.brandl
2007-04-04 23:44:52terry.reedycreate