Title: metaclasses, __getattr__, and special methods
Type: Stage:
Components: Interpreter Core Versions:
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: arigo, bpettersen, georg.brandl, mwh
Priority: normal Keywords:

Created on 2003-04-29 23:57 by bpettersen, last changed 2005-12-26 17:13 by arigo. This issue is now closed.

Messages (4)
msg15766 - (view) Author: Bjorn Pettersen (bpettersen) Date: 2003-04-29 23:57
__getattr__ on metaclasses aren't called when it would 
seem "logical" <wink> for it to be. E.g.:

>>> class meta(type):
...   def __getattr__(cls, name):
...     if name == '__len__':
...        print "meta.__getattr__('__len__')"
...        return lambda: 42
...     else:
...        print 'meta.__getattr__', name
...        return name
>>> class S(object):
...   __metaclass__ = meta
>>> S.__len__()
>>> len(S)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: len() of unsized object

I was told that special method "foo(x, arg)" was 
implemented as
"type(x).__foo__(x, arg)", which doesn't seem to be the 
case always...

>>> class meta(type):
...   def __len__(cls):
...     return 42
>>> class S(object):
...   __metaclass__ = meta
>>> S.__len__()
>>> len(S)

So, it looks like it's looking up __len__ in the 
metaclass, but not
falling back on __getattr__ when it isn't there? I've 
looked at the C
code and it seems like special methods each have their 
own way of
finding the function they're needing.

From Alex Martelli:
Ah yes, I see now!  Yes, functions such as len() rely on 
slots in
the type object, e.g. as you've noticed:

> finding the function they're needing, e.g. for len, it 
looks like it
> uses:
>   m = o->ob_type->tp_as_sequence;
>   if (m && m->sq_length)
>     return m->sq_length(o);
>   return PyMapping_Size(o);

and the "incredibly complex thinking" (quoting from 
in update_one_slot doesn't seem to work except for 
operations the
which "the class overrides in its dict" (again from a 
comment in
typeobject.c, this one for fixup_slot_dispatchers).

The issue may be with _PyType_Lookup (again in the 
same ,c file),
which just gives up if it can't find a name somewhere 
along the mro
(it doesn't "look upwards" to the metaclass) while 
DOES work upwards on the metaclass too.  Hmmmm.  
I'm not sure I
really understand all that's going on here - it IS a rather 
zone of the code.  Maybe you can post this as a bug in 
2.3 beta 1
on sourgeforge (ideally showing where in the docs it 
defines the
semantics that it then doesn't respect) so we can get 
this looked
at by the few people who really DO grasp these parts...;-
).  There
is probably some sound implementation reason for the 
behavior, but if so it should be better documented, I 

Back to me:
The point being that I haven't found any place in the 
documentation that defines what the attribute lookup is 
on new-style classes (and the C code is too hairy for 
me to understand :-)

As a special case of this problem, super() can't create 
an object which intercepts the special methods like it 
does for other methods, e.g.:

  super(MyCls, self).__getitem__(5)

works, but not

  super(MyCls, self)[5]

I don't know if that is intended or not, but it's not 
documented, though neither is exactly _what_ super is? 
(i.e. it looks like it's an object, that when you call a 
method, 'm', on it, uses the superclass method 'm', but 
the subclass versions of all other methods, although as 
above, not in all contexts, and I'm not sure whether 
you're supposed to be able to treat it as a first class 
object [pass as arg, return, etc])....

-- bjorn

msg15767 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2004-11-15 07:20
Logged In: YES 

You could try

(or attach pdf to the end instead...)

You say:

> The point being that I haven't found any place in the 
> documentation that defines what the attribute lookup is 
> on new-style classes

That's not the problem -- attribute lookup is fairly easy.  What 
you're missing is that attribute lookup != special method lookup.

This probably should be in the core documentation, yes.
msg15768 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2005-10-01 13:43
Logged In: YES 

I closed #789262 as a duplicate of this one. More info may
be there.
msg15769 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2005-12-26 17:13
Logged In: YES 

This is a known documentation bug: all this is expected, but
under-documented.  Indeed, len(x) calls the special method
__len__ of 'x', but what is not specified is the real
definition of "calling a special method" on an object 'x':
it is to look up the name "__len__" in the dict of type(x),
then in the dict of the parent types in MRO order.  It's
really not the same thing as an attribute lookup.
Date User Action Args
2003-04-29 23:57:21bpettersencreate