This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: getattr does not work well with descriptor
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Martin.Thurau, eric.snow, ncoghlan
Priority: normal Keywords:

Created on 2014-03-07 12:46 by Martin.Thurau, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
python_descriptor_bug.py Martin.Thurau, 2014-03-07 12:46 Simple script that demonstrates the problem with @property as well as a custom descriptor
Messages (4)
msg212876 - (view) Author: Martin Thurau (Martin.Thurau) Date: 2014-03-07 12:46
If you have a descriptor (in my case it was an SQLAlchemy column) on an instance and this descriptor returns None for a call to __get__ then getattr with a given default value, will not return the default, but None.

I have no knowledge on the implementation details of getattr but I guess the logic is something like this:
- getattr looks at the given object and sees that the attribute in question is not None (since it is the descriptor object)
- getattr returns the descriptor
- the descriptors __get__ method is called
- __get__ return None

Maybe it should be more like this:
- getattr looks at the given object and sees that the attribute in question is not None (since it is the descriptor object)
- getattr sees that the attribute has __get__
- getattr calls __get__ method and looks if the return value is None

I'm not sure if this is really a bug but it's highly confusing and somewhat un-pythonic. I really should not care of an attribute of an object is a value or a descriptor. This is especially true since this problem also applies to @property. Effectively this means that if you call getattr you have *know* if the name in question is a property or not and one can't simply swap out an objects value for a property without risking to break calling code.

If this is actually *not* a bug, we should at least update the documentation to getattr, to mention this fact. Because currently it states that "getattr(x, 'foobar') is equivalent to x.foobar" which is obviously not true.
msg212913 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-03-08 00:35
Returning None is the right thing here.  The default for getattr() is returned only when it catches an AttributeError (hence the exception is the sentinel, so to speak, not None.  Here's a rough equivalent:

  _notset = object()

  def getattr(obj, name, default=_notset):
      getter = type(obj).__getattribute__
      try:
          getter(obj, name)  # The normal object lookup machinery.
      except AttributeError:
          if default is _notset:
              raise
          return default

The underlying lookup machinery isn't the simplest thing to grok.  Here is the gist of what happens:

1. Try a data descriptor.
2. Try the object's __dict__.          
3. Try a non-data descriptor.
4. Try __getattr__().
5. raise AttributeError.      

Thus the descriptor's __get__() would have to raise AttributeError to trigger the default.  If need be, it should turn None into AttributeError or you can use some other default than None in your getattr() call and turn None into an AttributeError yourself.

What about "getattr(x, 'foobar') is equivalent to x.foobar" is not correct?  It doesn't matter if the value came from a descriptor's __get__ or from the object's __dict__.
msg212914 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-03-08 00:38
You may get unexpected behavior when you have a descriptor on a class that also has __getattr__ defined.  See issue #1615.  However, I don't think that applies here.  As far as I can tell, everything is working the way it should.
msg212922 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-03-08 04:41
Indeed, since None is a potentially valid attribute value, the required API for a descriptor to indicate "no such attribute" in __get__ is to throw AttributeError.
History
Date User Action Args
2022-04-11 14:57:59adminsetgithub: 65063
2014-03-08 04:41:13ncoghlansetstatus: open -> closed

nosy: + ncoghlan
messages: + msg212922

resolution: not a bug
stage: resolved
2014-03-08 00:38:10eric.snowsetmessages: + msg212914
2014-03-08 00:35:22eric.snowsetnosy: + eric.snow
messages: + msg212913
2014-03-07 12:46:12Martin.Thuraucreate