classification
Title: "inspect" gets broken by some descriptors
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, amaury.forgeotdarc, dmaurer, eric.araujo, eric.snow, facundobatista, flub, gvanrossum, jotr, meador.inge, pitrou, python-dev, vpelletier
Priority: normal Keywords: patch

Created on 2008-01-10 17:39 by dmaurer, last changed 2012-01-18 16:46 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
inspectBug.py dmaurer, 2008-01-10 17:39
inspect-bug.patch pitrou, 2008-01-10 23:25
inspect-and-pydoc-bug.patch pitrou, 2008-01-11 20:20
inspect-and-pydoc-bug2.patch pitrou, 2011-12-16 09:58
inspect-and-pydoc-bug3.patch pitrou, 2011-12-16 10:05
Messages (20)
msg59675 - (view) Author: Dieter Maurer (dmaurer) Date: 2008-01-10 17:39
The inspect functions "getmembers(cls)" and "classify_class_attrs(cls)"
require that for a class *cls* each name in "dir(cls)" can be retrieved
by "getattr(cls, name)". While this holds for usual class attributes, it
may well fail for descriptors (descriptors set by 'zope.interface' are a
real world example for this). Attached it as small script that
demonstrates the problem.

The bug affects 'pydoc' and the built in 'help' (which is in fact
'pydoc.help'). While 'pydoc' and 'help' do not break completely, they
can not present meaningful information for classes with some descriptors
msg59676 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-01-10 17:58
Please submit a patch.
msg59677 - (view) Author: Dieter Maurer (dmaurer) Date: 2008-01-10 19:18
In "dm.zdoc" (a "pydoc" wrapper for Zope) I simply filter out all names
returned by "dir" which cannot be "getattr"ed.
But, I am not sure, that this is good enough to be accepted as a patch
(although it already improves upon the current state)
msg59684 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-01-10 23:25
This is my attempt at a patch for this. It fixes inspect.getmembers and
inspect.classify_class_attrs to work with Dieter's example. I hope it
doesn't mess anything else.

As for pydoc, things look a bit more complicated... The annoying thing
is that the logic to browse object contents is duplicated inside the
different renderers (pydoc.Repr subclasses). Also, I couldn't find any
unit tests for pydoc. Are there any?
msg59712 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-01-11 14:54
For the record the same problem also happens with toscawidgets.

>>> from toscawidgets.widgets.forms import validators
>>> help(validators)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/site.py", line 351, in __call__
    return pydoc.help(*args, **kwds)
  File "/usr/lib/python2.5/pydoc.py", line 1646, in __call__
    self.help(request)
  File "/usr/lib/python2.5/pydoc.py", line 1690, in help
    else: doc(request, 'Help on %s:')
  File "/usr/lib/python2.5/pydoc.py", line 1481, in doc
    pager(title % desc + '\n\n' + text.document(object, name))
  File "/usr/lib/python2.5/pydoc.py", line 324, in document
    if inspect.ismodule(object): return self.docmodule(*args)
  File "/usr/lib/python2.5/pydoc.py", line 1072, in docmodule
    contents.append(self.document(value, key, name))
  File "/usr/lib/python2.5/pydoc.py", line 325, in document
    if inspect.isclass(object): return self.docclass(*args)
  File "/usr/lib/python2.5/pydoc.py", line 1173, in docclass
    classify_class_attrs(object))
  File "/usr/lib/python2.5/pydoc.py", line 179, in classify_class_attrs
    return map(fixup, inspect.classify_class_attrs(object))
  File "/usr/lib/python2.5/inspect.py", line 246, in classify_class_attrs
    obj = getattr(cls, name)
  File
"/usr/lib/python2.5/site-packages/FormEncode-0.7.1-py2.5.egg/formencode/declarative.py",
line 105, in __get__
    obj = type.singleton()
  File
"/usr/lib/python2.5/site-packages/FormEncode-0.7.1-py2.5.egg/formencode/declarative.py",
line 166, in singleton
    setattr(cls, name, cls(declarative_count=cls.declarative_count))
TypeError: __init__() got an unexpected keyword argument 'declarative_count'
msg59743 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-01-11 20:20
Here is a (hopefully complete) patch against both pydoc and inspect. It
fixes one more bug compared to the previous one (descriptors can also
have a special __getattr__ in addition to __get__, which gave problems
when trying to access __classobj__).
msg67777 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2008-06-06 16:17
Really don't know why this was assigned to me...
msg116863 - (view) Author: Mark Lawrence (BreamoreBoy) Date: 2010-09-19 11:15
I can't apply the patch to any current SVN version.
msg149242 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2011-12-11 19:24
The 'type' object now has the same issue: __abstractmethods__ appears in dir(type) but type.__abstractmethods__ fails with an AttributeError.
See issue13581
msg149606 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-16 09:58
Updated patch for 3.2.
msg149607 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-16 10:05
Updated patch that adds a test for classifying builtin types (checks that ``inspect.classify_class_attrs(type)`` does not throw as in issue13581).
msg149968 - (view) Author: Roundup Robot (python-dev) Date: 2011-12-21 09:01
New changeset 902f694a7b0e by Antoine Pitrou in branch '3.2':
Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
http://hg.python.org/cpython/rev/902f694a7b0e

New changeset b08bf8df8eec by Antoine Pitrou in branch 'default':
Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
http://hg.python.org/cpython/rev/b08bf8df8eec
msg149970 - (view) Author: Roundup Robot (python-dev) Date: 2011-12-21 09:18
New changeset 13f56cd8dec1 by Antoine Pitrou in branch '2.7':
Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
http://hg.python.org/cpython/rev/13f56cd8dec1
msg149973 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-21 09:19
Now fixed in all 3 branches.
msg150005 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-12-21 16:10
Great stuff.
msg151544 - (view) Author: Vincent Pelletier (vpelletier) Date: 2012-01-18 15:47
This change causes the following behaviour:

>>> import inspect
>>> class B(object):
...   def f(self):
...     pass
... 
>>> inspect.getmembers(B, inspect.ismethod)
[]

While I would expect the result to contain f:

>>> inspect.ismethod(B.f)
True

Isn't this a regression ?

Regards,
Vincent Pelletier
msg151545 - (view) Author: Vincent Pelletier (vpelletier) Date: 2012-01-18 15:49
Sorry, I forgot to mention I'm using python2.7 .
msg151548 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-01-18 16:29
Thanks for noticing. The doc for ismethod() says:

  “Return true if the object is a bound method written in Python.”

and the docstring agrees with that:

  “Return true if the object is an instance method. [...]”

So the change isn't properly a regression when reading the docs. On the other hand, it's true that some code may rely on the previous behaviour, and the discrepancy between getmembers() and a manual test can be confusing.

By the way, Python 3 has ismethod() right:

>>> class B:
...   def f(self): pass
... 
>>> inspect.ismethod(B.f)
False
>>> inspect.ismethod(B().f)
True
msg151549 - (view) Author: Roundup Robot (python-dev) Date: 2012-01-18 16:41
New changeset f824744557ba by Antoine Pitrou in branch '2.7':
Revert part of 13f56cd8dec1 (issue #1785) to avoid breaking getmembers() with unbound methods.
http://hg.python.org/cpython/rev/f824744557ba
msg151550 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-01-18 16:46
I've backed out the part of the changeset that "fixed" getmembers(), so the old behaviour is restored. Other parts of the changeset (that e.g. fixed pydoc) have not been reverted.
History
Date User Action Args
2012-01-18 16:46:09pitrousetmessages: + msg151550
versions: + Python 2.7, Python 3.3
2012-01-18 16:41:03python-devsetmessages: + msg151549
2012-01-18 16:29:58pitrousetmessages: + msg151548
2012-01-18 15:49:43vpelletiersetmessages: + msg151545
2012-01-18 15:47:31vpelletiersetnosy: + vpelletier
messages: + msg151544
2011-12-21 16:10:25eric.araujosetnosy: + eric.araujo
messages: + msg150005
2011-12-21 09:19:07pitrousetstatus: open -> closed
resolution: fixed
messages: + msg149973

stage: patch review -> resolved
2011-12-21 09:18:54pitroulinkissue13581 superseder
2011-12-21 09:18:05python-devsetmessages: + msg149970
2011-12-21 09:01:09python-devsetnosy: + python-dev
messages: + msg149968
2011-12-16 17:28:04eric.snowsetnosy: + eric.snow
2011-12-16 10:05:43pitrousetfiles: + inspect-and-pydoc-bug3.patch
2011-12-16 10:05:33pitrousetmessages: + msg149607
2011-12-16 09:58:38pitrousetfiles: + inspect-and-pydoc-bug2.patch

stage: patch review
messages: + msg149606
versions: + Python 3.2, - Python 2.6
2011-12-13 02:12:20meador.ingesetnosy: + meador.inge
2011-12-11 19:24:39amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg149242
2010-09-19 11:15:27BreamoreBoysetnosy: + BreamoreBoy
messages: + msg116863
2010-03-29 16:57:05jotrsetnosy: + jotr
2009-02-18 11:28:40flubsetnosy: + flub
2008-06-06 16:18:11facundobatistasetassignee: facundobatista ->
messages: + msg67777
2008-03-18 02:23:14jafosetpriority: normal
assignee: facundobatista
keywords: + patch
nosy: + facundobatista
2008-01-11 20:20:15pitrousetfiles: + inspect-and-pydoc-bug.patch
messages: + msg59743
2008-01-11 14:54:15pitrousetmessages: + msg59712
2008-01-10 23:25:32pitrousetversions: + Python 2.6, - Python 2.5
2008-01-10 23:25:25pitrousetfiles: + inspect-bug.patch
nosy: + pitrou
messages: + msg59684
2008-01-10 19:18:18dmaurersetmessages: + msg59677
2008-01-10 17:58:29gvanrossumsetnosy: + gvanrossum
messages: + msg59676
2008-01-10 17:39:24dmaurercreate