This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: inspect.getmembers passes exceptions from object's properties through
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: richardbruskiewich, rominf, xtreak, yselivanov
Priority: normal Keywords: patch

Created on 2018-10-30 08:20 by rominf, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 18330 closed tomaugspurger, 2020-02-03 15:21
Messages (5)
msg328885 - (view) Author: Roman Inflianskas (rominf) Date: 2018-10-30 08:20
I use inspect.getmembers for getting members of splinter.webdriver.BaseWebDriver. The problem is that it has a property status_code raises NotImplementedError: https://github.com/cobrateam/splinter/blob/master/splinter/driver/webdriver/__init__.py#L191. This exception passes through try block in https://github.com/python/cpython/blob/master/Lib/inspect.py#L343 because it doesn't contain:
except Exception:
    pass
section.
In the result, instead of members of the object, I get an exception.

I think there are two possible expected behaviors:
1. Just ignore the exceptions and return members without the members, that raise exceptions.
2. Instead of members, return tuple like:
...
try:
    ...
except Exception as e:
    RAISES_EXCEPTION = object()
    value = (RAISES_EXCEPTION, e)
...
msg328890 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-30 09:47
Thanks for the report. A simple reproducible in Python without splinter code. I think it happens for instances of the class where the attribute is a property raising exception and executing inspect.getmembers(Foo) works fine.

# bpo35108.py

import inspect

class Foo():

    @property
    def bar(self):
        raise NotImplementedError

print(inspect.getmembers(Foo()))

$. /python.exe ../backups/bpo35108.py
Traceback (most recent call last):
  File "../backups/bpo35108.py", line 9, in <module>
    print(inspect.getmembers(Foo()))
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 344, in getmembers
    value = getattr(object, key)
  File "../backups/bpo35108.py", line 7, in bar
    raise NotImplementedError
NotImplementedError
msg328891 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-30 10:11
Sorry I misread your title stating it's about object getattr for a property has to execute the underlying code it seems as explained in https://stackoverflow.com/a/30143990/2610955 . I don't know if there is a way through which an attribute access in case of property can be verified without executing it. In case of catching the exception and returning the attributes that don't raise exception since it depends on the runtime execution of the property where the exception might occur at one point in time and doesn't occur in another point of time causing confusion during debugging.
msg329254 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-11-04 20:22
There are some relevant docs in the inspect module about properties triggering code execution and thus using getattr_static to fetch attributes. Ref : https://docs.python.org/3/library/inspect.html#fetching-attributes-statically . I tried the below and it passes for your code but there might be still cases where getattr_static triggers an exception too where we need to decide whether to skip the attribute from being listed and if so with a test for the scenario.

Using getattr_static first causes test_inspect.TestPredicates.test_get_slot_members to fail by including slot_member. Thus we try for getattr and if it raises an exception that is not AttributeError (@property might also raise AttributeError consciously at runtime) and then use getattr_static in case of exception other than AttributeError. This helps in passing the test suite and also the example attached listing bar which raises NotImplementedError or other Exceptions.


diff --git a/Lib/inspect.py b/Lib/inspect.py
index b8a142232b..9df2173e0c 100644
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -341,7 +341,12 @@ def getmembers(object, predicate=None):
         # like calling their __get__ (see bug #1785), so fall back to
         # looking in the __dict__.
         try:
-            value = getattr(object, key)
+            try:
+                value = getattr(object, key)
+            except Exception as e:
+                if isinstance(e, AttributeError):
+                    raise e
+                value = getattr_static(object, key)
             # handle the duplicate key
             if key in processed:
                 raise AttributeError


I am adding Yury. Removing 3.5 and 3.4 since they are in security fixes only mode.

Also see issue30533 a proposal for implementation of getmembers that uses getattr_static instead of getattr.


Thanks
msg361208 - (view) Author: Richard Bruskiewich (richardbruskiewich) Date: 2020-02-02 04:46
This "bug" is buzzing around my project head right now, interfering with the operation of the Python Fire CLI library when it attempts to interrogate the Python Pandas DataFrame using the inspect.getmembers() call. See https://github.com/pandas-dev/pandas/issues/31474 and https://github.com/pandas-dev/pandas/pull/31549.

I have code that uses Fire and Pandas, but have to "dumb it down" to use Pandas 0.24.3 rather than the latest 0.25.1 which raises a "NotImplementedError" which leaks out of the getmembers() call.  The Pandas people are passing the buck to you folks in the Python community.

This is terribly frustrating for we minions in the real work trying to implementing real working software systems leveraging all these wonderful libraries (and the Python language).

When is this "bug" going to be fixed? Help!
History
Date User Action Args
2022-04-11 14:59:07adminsetgithub: 79289
2020-02-03 15:21:31tomaugspurgersetkeywords: + patch
stage: patch review
pull_requests: + pull_request17703
2020-02-02 04:46:38richardbruskiewichsetnosy: + richardbruskiewich
messages: + msg361208
2018-11-04 20:22:21xtreaksetnosy: + yselivanov

messages: + msg329254
versions: - Python 3.4, Python 3.5
2018-10-30 10:11:54xtreaksetmessages: + msg328891
2018-10-30 09:47:22xtreaksetnosy: + xtreak
messages: + msg328890
2018-10-30 08:20:35rominfcreate