classification
Title: Wrong AttributeError propagation
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.4, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: dunric, eryksun, martin.panter
Priority: normal Keywords:

Created on 2015-09-02 10:14 by dunric, last changed 2015-09-02 19:20 by dunric. This issue is now closed.

Messages (6)
msg249533 - (view) Author: David Unric (dunric) Date: 2015-09-02 10:14
Hello,

it seems python interpreter improperly handles AttributeError exception raised in __getattr__ method, after called by unresolved attribute inside a property.

Bellow is a simple Python2 example of a class which defines __getattr__ method and a property, where is non-existing attribute accessed:

from __future__ import print_function

class MyClass(object):
    def __getattr__(self, name):
        print('__getattr__ <<', name)
        raise AttributeError(name)
        return 'need know the question'

    @property
    def myproperty(self):
        print(self.missing_attribute)
        return 42

my_inst = MyClass()
print(my_inst.myproperty)

# produces following output
__getattr__ << missing_attribute
__getattr__ << myproperty
Traceback (most recent call last):
  File "a.py", line 84, in <module>
    main()
  File "a.py", line 74, in main
    print('==', my_inst.myproperty)
  File "a.py", line 36, in __getattr__
    raise AttributeError(name)
AttributeError: myproperty


By the documentation https://docs.python.org/2/reference/datamodel.html#object.__getattr__ , if class defines __getattr__ method, it gets called at AttributeError exception and should return a computed value for name, or raise (new) AttributeError exception.

The misbehavior is in 2nd call of __getattr__, with 'myproperty' as an argument.

- self.myproperty does exist, no reason to call __getattr__ for it
- AttributeError exception raised in __getattr__ should be propagated to the outer stack, no recurrence in the same
msg249534 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-09-02 10:40
First of all, I think your a.py line 74 corresponds to the top-level print() call. My version of the output:

>>> print(my_inst.myproperty)
__getattr__ << missing_attribute
__getattr__ << myproperty
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in __getattr__
AttributeError: myproperty

This is a bit tricky, but I _think_ this is working as it is meant to. I think your central problem is an unhandled AttributeError is leaking:

1. Evaluating “my_inst.myproperty” invokes your myproperty() getter method.

2. That function tries to evaluate “self.missing_attribute”, which invokes your __getattr__() method.

3. __getattr__() raises AttributeError for “missing_attribute”.

4. myproperty() does not handle the AttributeError, so it leaks. I’m not sure if the documentation is clear on this, but a property getter raising AttributeError signals that the property attribute does not exist, even though this AttributeError was originally referring to “missing_attribute”.

5. Python thinks that the “myproperty” property doesn’t exist, so it falls back to __getattr__().

6. __getattr__() raises AttributeError for “myproperty”.

Hopefully that makes sense and you can see it is working somewhat sensibly :)
msg249537 - (view) Author: David Unric (dunric) Date: 2015-09-02 11:13
Thanks for the comprehensive response.

I did suspected the 2nd call is caused how it has been described in paragraph 4. And you are probably right. Only think exception instance raised in __getattr__ should not lead to its another call but propagated to outer level, ie. should not get continue in getter in this example.
msg249538 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-09-02 11:46
> exception instance raised in __getattr__ should not lead 
> to its another call but propagated to outer level

In this case the next outer level is your descriptor implementation. You have to ensure it doesn't leak the AttrubuteError. Otherwise the associated __getattribute__ call is implicitly raising AttributeError, and [by design][1] Python must default to calling __getattr__.

[1]: https://docs.python.org/3/reference/datamodel.html#object.__getattribute__
msg249559 - (view) Author: David Unric (dunric) Date: 2015-09-02 18:47
This looks a bit inconsistent. See following version with "manual" getter definition:

class MyClass(object):
    def __init__(self, *args, **kwargs):
        # setting access to getter by attribute
        # without use of property decorator
        self.__dict__['myproperty'] = self._myproperty()

    def __getattr__(self, name):
        print('__getattr__ <<', name)
        raise AttributeError(name)
        return 'need know the question'

    def _myproperty(self):
        print(self.missing_attribute)
        return 42

my_inst = MyClass()
print(my_inst.myproperty)

# Produces (expected) output
__getattr__ << missing_attribute
Traceback (most recent call last):
  File "a.py", line 55, in <module>
    my_inst = MyClass()
  File "a.py", line 33, in __init__
    self.__dict__['myproperty'] = self._myproperty()
  File "a.py", line 48, in _myproperty
    print(self.missing_attribute)
  File "a.py", line 37, in __getattr__
    raise AttributeError(name)
AttributeError: missing_attribute


I'd expect the original version using property decorator would behave the same way. Possible issue in property class implementation ?
msg249562 - (view) Author: David Unric (dunric) Date: 2015-09-02 19:20
Oops, this was the strict version. The lazy method call version behaves exactly like property getter.

So there is probably no implementation bug, but not too well thought out decision design, making debugging AttributeError exceptions in properties difficult and quite misleading.
History
Date User Action Args
2015-09-02 19:20:10dunricsetmessages: + msg249562
2015-09-02 18:47:09dunricsetmessages: + msg249559
2015-09-02 11:46:21eryksunsetnosy: + eryksun

messages: + msg249538
stage: resolved
2015-09-02 11:13:54dunricsetmessages: + msg249537
2015-09-02 10:40:13martin.pantersetstatus: open -> closed

nosy: + martin.panter
messages: + msg249534

resolution: not a bug
2015-09-02 10:14:48dunriccreate