Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inspect.getmembers passes exceptions from object's properties through #79289

Open
rominf mannequin opened this issue Oct 30, 2018 · 6 comments
Open

inspect.getmembers passes exceptions from object's properties through #79289

rominf mannequin opened this issue Oct 30, 2018 · 6 comments
Labels
3.12 bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@rominf
Copy link
Mannequin

rominf mannequin commented Oct 30, 2018

BPO 35108
Nosy @1st1, @tirkarthi, @RichardBruskiewich
PRs
  • bpo-35108: handle exceptions in inspect.getmembers #18330
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2018-10-30.08:20:35.443>
    labels = ['3.7', '3.8', 'type-bug', 'library']
    title = "inspect.getmembers passes exceptions from object's properties through"
    updated_at = <Date 2020-02-03.15:21:31.209>
    user = 'https://bugs.python.org/rominf'

    bugs.python.org fields:

    activity = <Date 2020-02-03.15:21:31.209>
    actor = 'tomaugspurger'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2018-10-30.08:20:35.443>
    creator = 'rominf'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 35108
    keywords = ['patch']
    message_count = 5.0
    messages = ['328885', '328890', '328891', '329254', '361208']
    nosy_count = 4.0
    nosy_names = ['yselivanov', 'rominf', 'xtreak', 'richardbruskiewich']
    pr_nums = ['18330']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue35108'
    versions = ['Python 2.7', 'Python 3.6', 'Python 3.7', 'Python 3.8']

    @rominf
    Copy link
    Mannequin Author

    rominf mannequin commented Oct 30, 2018

    I use inspect.getmembers for getting members of splinter.webdriver.BaseWebDriver. The problem is that it has a property status_code raises NotImplementedError: https://github.com/cobrateam/splinter/blob/master/splinter/driver/webdriver/__init__.py#L191. This exception passes through try block in https://github.com/python/cpython/blob/master/Lib/inspect.py#L343 because it doesn't contain:
    except Exception:
    pass
    section.
    In the result, instead of members of the object, I get an exception.

    I think there are two possible expected behaviors:

    1. Just ignore the exceptions and return members without the members, that raise exceptions.
    2. Instead of members, return tuple like:
      ...
      try:
      ...
      except Exception as e:
      RAISES_EXCEPTION = object()
      value = (RAISES_EXCEPTION, e)
      ...

    @rominf rominf mannequin added 3.8 only security fixes 3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Oct 30, 2018
    @tirkarthi
    Copy link
    Member

    Thanks for the report. A simple reproducible in Python without splinter code. I think it happens for instances of the class where the attribute is a property raising exception and executing inspect.getmembers(Foo) works fine.

    # bpo-35108.py

    import inspect
    
    class Foo():
    
        @property
        def bar(self):
            raise NotImplementedError
    
    print(inspect.getmembers(Foo()))
    $. /python.exe ../backups/bpo35108.py
    Traceback (most recent call last):
      File "../backups/bpo35108.py", line 9, in <module>
        print(inspect.getmembers(Foo()))
      File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 344, in getmembers
        value = getattr(object, key)
      File "../backups/bpo35108.py", line 7, in bar
        raise NotImplementedError
    NotImplementedError

    @tirkarthi
    Copy link
    Member

    Sorry I misread your title stating it's about object getattr for a property has to execute the underlying code it seems as explained in https://stackoverflow.com/a/30143990/2610955 . I don't know if there is a way through which an attribute access in case of property can be verified without executing it. In case of catching the exception and returning the attributes that don't raise exception since it depends on the runtime execution of the property where the exception might occur at one point in time and doesn't occur in another point of time causing confusion during debugging.

    @tirkarthi
    Copy link
    Member

    There are some relevant docs in the inspect module about properties triggering code execution and thus using getattr_static to fetch attributes. Ref : https://docs.python.org/3/library/inspect.html#fetching-attributes-statically . I tried the below and it passes for your code but there might be still cases where getattr_static triggers an exception too where we need to decide whether to skip the attribute from being listed and if so with a test for the scenario.

    Using getattr_static first causes test_inspect.TestPredicates.test_get_slot_members to fail by including slot_member. Thus we try for getattr and if it raises an exception that is not AttributeError (@Property might also raise AttributeError consciously at runtime) and then use getattr_static in case of exception other than AttributeError. This helps in passing the test suite and also the example attached listing bar which raises NotImplementedError or other Exceptions.

    diff --git a/Lib/inspect.py b/Lib/inspect.py
    index b8a142232b..9df2173e0c 100644
    --- a/Lib/inspect.py
    +++ b/Lib/inspect.py
    @@ -341,7 +341,12 @@ def getmembers(object, predicate=None):
             # like calling their __get__ (see bug python/cpython#46118), so fall back to
             # looking in the __dict__.
             try:
    -            value = getattr(object, key)
    +            try:
    +                value = getattr(object, key)
    +            except Exception as e:
    +                if isinstance(e, AttributeError):
    +                    raise e
    +                value = getattr_static(object, key)
                 # handle the duplicate key
                 if key in processed:
                     raise AttributeError

    I am adding Yury. Removing 3.5 and 3.4 since they are in security fixes only mode.

    Also see bpo-30533 a proposal for implementation of getmembers that uses getattr_static instead of getattr.

    Thanks

    @RichardBruskiewich
    Copy link
    Mannequin

    RichardBruskiewich mannequin commented Feb 2, 2020

    This "bug" is buzzing around my project head right now, interfering with the operation of the Python Fire CLI library when it attempts to interrogate the Python Pandas DataFrame using the inspect.getmembers() call. See pandas-dev/pandas#31474 and pandas-dev/pandas#31549.

    I have code that uses Fire and Pandas, but have to "dumb it down" to use Pandas 0.24.3 rather than the latest 0.25.1 which raises a "NotImplementedError" which leaks out of the getmembers() call. The Pandas people are passing the buck to you folks in the Python community.

    This is terribly frustrating for we minions in the real work trying to implementing real working software systems leveraging all these wonderful libraries (and the Python language).

    When is this "bug" going to be fixed? Help!

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @iritkatriel
    Copy link
    Member

    @tirkarthi Perhaps NotImplementedError can be handled as a special case? It indicates that the property is not really there, so it should be ok to omit it from the output.

    @iritkatriel iritkatriel added 3.12 bugs and security fixes and removed 3.8 only security fixes 3.7 (EOL) end of life labels Jul 8, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.12 bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants