Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Display exceptions' subclasses in help() #52771

Closed
robcliffe mannequin opened this issue Apr 24, 2010 · 26 comments
Closed

Display exceptions' subclasses in help() #52771

robcliffe mannequin opened this issue Apr 24, 2010 · 26 comments
Assignees
Labels
3.8 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@robcliffe
Copy link
Mannequin

robcliffe mannequin commented Apr 24, 2010

BPO 8525
Nosy @birkenfeld, @terryjreedy, @ncoghlan, @abalkin, @merwok, @briancurtin, @serhiy-storchaka, @eric-wieser, @CuriousLearner, @tirkarthi
PRs
  • bpo-8525: help() on a type now shows builtin subclasses #5066
  • Files
  • pydoc.py: pydoc.py with enhancement to help()
  • help_8525.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/CuriousLearner'
    closed_at = <Date 2018-10-21.07:27:06.464>
    created_at = <Date 2010-04-24.23:10:00.456>
    labels = ['3.8', 'type-feature', 'library']
    title = "Display exceptions' subclasses in help()"
    updated_at = <Date 2018-12-29.15:22:31.762>
    user = 'https://bugs.python.org/robcliffe'

    bugs.python.org fields:

    activity = <Date 2018-12-29.15:22:31.762>
    actor = 'serhiy.storchaka'
    assignee = 'CuriousLearner'
    closed = True
    closed_date = <Date 2018-10-21.07:27:06.464>
    closer = 'ncoghlan'
    components = ['Library (Lib)']
    creation = <Date 2010-04-24.23:10:00.456>
    creator = 'robcliffe'
    dependencies = []
    files = ['17075', '19748']
    hgrepos = []
    issue_num = 8525
    keywords = ['patch']
    message_count = 26.0
    messages = ['104134', '104656', '104657', '104658', '121940', '122044', '122057', '122115', '122116', '122118', '122193', '123167', '123169', '123170', '123228', '288243', '309296', '321651', '321675', '322226', '322240', '322287', '322288', '328194', '328195', '332721']
    nosy_count = 14.0
    nosy_names = ['georg.brandl', 'terry.reedy', 'ncoghlan', 'belopolsky', 'ron_adam', 'eric.araujo', 'brian.curtin', 'robcliffe', 'henriquebastos', 'Rodolpho.Eckhardt', 'serhiy.storchaka', 'Eric.Wieser', 'CuriousLearner', 'xtreak']
    pr_nums = ['5066']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue8525'
    versions = ['Python 3.8']

    @robcliffe
    Copy link
    Mannequin Author

    robcliffe mannequin commented Apr 24, 2010

    help() on an exception class lists the method resolution order, which is in effect the class inheritance hierarchy. E.g. help(ArithmeticError) lists ArithmeticError, StandardError, Exception, BaseException, __builtin__.object. It struck me it would help to find my way around if it also listed the builtin SUBclasses (if any). Something like:
    Built-in subclasses:
    FloatingPointError
    OverflowError
    ZeroDivisionError
    In fact why not do it for any class, not just exceptions?
    I attach a patched version of pydoc.py - tested but only on my PC which is running Python 2.5 under Windows XP. I have added lines 1129-1148 to the docclass method of the TextDoc class (and flagged them # [RAC] ).
    (I don't pretend to understand the magic where __builtins__ is a dictionary when pydoc.py is run but becomes a module later on. Never mind - the patch works (I believe).)
    For consistency, a similar patch would also have to be made to the docclass nethod of the HTMLDoc class (which outputs HTML rather than plain text). I have not attempted this as I don't know how it is called and hence how to test any patch, but it should be straightforward for anyone with the know-how.

    @robcliffe robcliffe mannequin added the type-feature A feature request or enhancement label Apr 24, 2010
    @birkenfeld birkenfeld self-assigned this Apr 25, 2010
    @terryjreedy
    Copy link
    Member

    2.5 is frozen. 2.6 and 3.1 are in bug-fix mode only, and 2.7 is in beta so new features are unlikely there. So I recommend doing a patch against 3.1 or even the 3.2 trunk. And of course, make sure this proposal makes sense for 3.x.

    @merwok
    Copy link
    Member

    merwok commented Apr 30, 2010

    Thanks for your work, Rob. To get reviews and comments, you’ll need to submit a patch (a diff) instead of the whole file. This makes it easier to see your changes than looking for a special comment :)

    This short doc should contain all the information you’ll need: http://www.python.org/dev/patches

    As Terry said, 2.6 and 3.1 only get bug fixes, 2.7 and 3.2 are in beta stage and don’t get new features, so you’ll want to port your changes to the py3k branch (the trunk for the 3.x series).

    Regards

    @merwok merwok added the stdlib Python modules in the Lib dir label Apr 30, 2010
    @briancurtin
    Copy link
    Member

    Minor correction to the last comment: 3.2 is not in beta nor feature freeze.

    @RodolphoEckhardt
    Copy link
    Mannequin

    RodolphoEckhardt mannequin commented Nov 21, 2010

    Migrated the patch to the 3.x trunk and added three tests.

    @merwok
    Copy link
    Member

    merwok commented Nov 22, 2010

    Thank you. I uploaded your patch to Rietveld and reviewed it: http://codereview.appspot.com/3169042/

    @abalkin
    Copy link
    Member

    abalkin commented Nov 22, 2010

    The following passes tests in elp_8525.patch, but is much simpler:

    ===================================================================

    --- Lib/pydoc.py	(revision 86600)
    +++ Lib/pydoc.py	(working copy)
    @@ -1139,6 +1139,15 @@
                     push('    ' + makename(base))
                 push('')
     
    +        # List the built-in subclasses, if any:
    +        subclasses = [cls.__name__ for cls in object.__subclasses__()
    +                      if cls.__module__ == 'builtins']
    +        if subclasses:
    +            push("Built-in subclasses:")
    +            for subclassname in sorted(subclasses):
    +                push('    ' + subclassname)
    +            push('')
    +
             # Cute little class to pump out a horizontal rule between sections.
             class HorizontalRule:
                 def __init__(self):

    @robcliffe
    Copy link
    Mannequin Author

    robcliffe mannequin commented Nov 22, 2010

    Thanks for your work. Glad if I have made a contribution to Python,
    however small.
    Rob Cliffe

    On 22/11/2010 00:26, Éric Araujo wrote:

    Éric Araujo<merwok@netwok.org> added the comment:

    Thank you. I uploaded your patch to Rietveld and reviewed it: http://codereview.appspot.com/3169042/

    ----------


    Python tracker<report@bugs.python.org>
    <http://bugs.python.org/issue8525\>


    @robcliffe
    Copy link
    Mannequin Author

    robcliffe mannequin commented Nov 22, 2010

    I would not be at all surprised if my patch could be simplified (in fact
    I'd be surprised if it couldn't).
    However, I did try out your version on Python 2.5 specifically, and it
    did not work for me.
    Trying it out on help(Exception), the relevant members of
    object.__subclasses__() viz.
    <type 'exceptions.StandardError'>, <type 'exceptions.StopIteration'> etc.
    had a __module__attribute which equalled 'exceptions', not 'builtins'.
    Best wishes
    Rob Cliffe

    On 22/11/2010 01:33, Alexander Belopolsky wrote:

    Alexander Belopolsky<belopolsky@users.sourceforge.net> added the comment:

    The following passes tests in elp_8525.patch, but is much simpler:

    ===================================================================
    --- Lib/pydoc.py (revision 86600)
    +++ Lib/pydoc.py (working copy)
    @@ -1139,6 +1139,15 @@
    push(' ' + makename(base))
    push('')

    •    # List the built-in subclasses, if any:
      
    •    subclasses = [cls.\_\_name__ for cls in object.\_\_subclasses__()
      
    •                  if cls.\_\_module__ == 'builtins']
      
    •    if subclasses:
      
    •        push("Built-in subclasses:")
      
    •        for subclassname in sorted(subclasses):
      
    •            push('    ' + subclassname)
      
    •        push('')
      
    •     # Cute little class to pump out a horizontal rule between sections.
          class HorizontalRule:
              def \_\_init__(self):
      

    ----------
    nosy: +belopolsky


    Python tracker<report@bugs.python.org>
    <http://bugs.python.org/issue8525\>


    @merwok
    Copy link
    Member

    merwok commented Nov 22, 2010

    New features can only go into 3.2, so you have to test with an updated checkout of the Subversion branch named py3k. See the link I have in a previous message.

    (P.S. Would you be so kind as to edit quoted text out of your replies? It’s unnecessary noise. Thanks in advance.)

    @abalkin
    Copy link
    Member

    abalkin commented Nov 23, 2010

    The proposal is to display builtin subclasses as for example:

    >>> help(ArithmeticError)
    class ArithmeticError(Exception)
     |  Base class for arithmetic errors.
     |  
     |  Method resolution order:
     |      ArithmeticError
     |      Exception
     |      BaseException
     |      object
     |  
     |  Built-in subclasses:
     |      FloatingPointError
     |      OverflowError
     |      ZeroDivisionError

    Note that this really affects only exceptions because no other builtin class has builtin subclasses. (dict has subclasses in collections, but not in builtins.)

    Exception hierarchy is presented in the reference manual at

    http://docs.python.org/dev/library/exceptions.html?#exception-hierarchy

    I wonder if rather than having MRO and subclasses sections, we should just present a portion of the exception hierarchy including the given exception, all its bases and direct subclasses:

    object
    |
    BaseException
    |
    Exception
    |
    *ArithmeticError*
    |
    +-- FloatingPointError
    +-- OverflowError
    +-- ZeroDivisionError

    @abalkin abalkin changed the title Small enhancement to help() Display exception's subclasses in help() Nov 23, 2010
    @merwok
    Copy link
    Member

    merwok commented Dec 3, 2010

    Didn’t the first message ask for the feature to be extended to non-exceptions classes? “Built-in” subclasses is a red herring, to me the feature is: display subclasses. In the text output you can use the search feature of your pager to jump to a subclass, in the HTML output it would be a helpful link.

    @abalkin
    Copy link
    Member

    abalkin commented Dec 3, 2010

    On Thu, Dec 2, 2010 at 9:56 PM, Éric Araujo <report@bugs.python.org> wrote:
    ..

    Didn’t the first message ask for the feature to be extended to non-exceptions classes?  “Built-in”
    subclasses is a red herring, to me the feature is: display subclasses.  In the text output you can use
    the search feature of your pager to jump to a subclass, in the HTML output it would be a helpful link.

    If we don't restrict to builtins, then the display will depend on what
    modules are currently loaded and will be useless for command line use
    unless all subclasses are defined in the same module.

    @merwok
    Copy link
    Member

    merwok commented Dec 3, 2010

    Yes, I imagine the feature to be useful to find subclasses defined in the same module. Rob, am I misunderstanding your original request?

    @robcliffe
    Copy link
    Mannequin Author

    robcliffe mannequin commented Dec 3, 2010

    Originally I only had built-in classes in mind (with Exceptions being
    IMO the most obvious example of a built-in class hierarchy that it is
    useful to find your way around). But if the idea can be extended to
    other classes, well, great.
    Rob Cliffe

    @birkenfeld birkenfeld removed their assignment Oct 6, 2012
    @CuriousLearner
    Copy link
    Member

    Hi,

    It seems that it hasn't been worked upon from quite a long time and the patch would also need changes. May I work on this?

    @CuriousLearner
    Copy link
    Member

    I've changed the version to 3.7. Not sure if this would need a backport since this is a new feature. So, I'll defer this call to a core developer. I've created a PR for the new feature.

    @CuriousLearner CuriousLearner added the 3.7 (EOL) end of life label Dec 31, 2017
    @CuriousLearner CuriousLearner added 3.8 only security fixes and removed 3.7 (EOL) end of life labels Feb 8, 2018
    @CuriousLearner CuriousLearner self-assigned this Feb 8, 2018
    @pppery pppery mannequin changed the title Display exception's subclasses in help() Display exceptions' subclasses in help() Feb 8, 2018
    @ncoghlan
    Copy link
    Contributor

    Reviewing the builtins in 3.7, I get the following results for builtin objects that have defined subclasses immediately after interpreter startup:

    =============

    >>> for name, obj in vars(builtins).items():
    ...     if isinstance(obj, type) and name in str(obj):
    ...         subclasses = type(obj).__subclasses__(obj)
    ...         if subclasses:
    ...             print(f"{obj}: {len(subclasses)}")
    ... 
    <class 'classmethod'>: 1
    <class 'dict'>: 1
    <class 'property'>: 1
    <class 'int'>: 1
    <class 'object'>: 132
    <class 'staticmethod'>: 1
    <class 'tuple'>: 16
    <class 'type'>: 1
    <class 'BaseException'>: 4
    <class 'Exception'>: 19
    <class 'ImportError'>: 2
    <class 'OSError'>: 13
    <class 'RuntimeError'>: 3
    <class 'NameError'>: 1
    <class 'SyntaxError'>: 1
    <class 'IndentationError'>: 1
    <class 'LookupError'>: 3
    <class 'ValueError'>: 2
    <class 'UnicodeError'>: 3
    <class 'ArithmeticError'>: 3
    <class 'SystemError'>: 1
    <class 'Warning'>: 10
    <class 'ConnectionError'>: 4

    =============

    So rather than special-casing exceptions or builtins in general, my inclination would be to include a section that lists up to 4 subclasses inline, and then adds a "... and NNN additional subclasses" trailing when there are more than 4 (or when there are less than 4 subclasses with public names, but additional private subclasses).

    So in a class like "int", for example, we'd see:

    Currently imported subclasses:
        bool
    

    While in a class like OSError we'd see:

    Currently imported subclasses:
        BlockingIOError
        ChildProcessError
        ConnectionError
        FileExistsError
        ... and 9 other subclasses
    

    And in "help(object)" we'd see something like:

    Currently imported subclasses:
        async_generator
        BaseException
        builtin_function_or_method
        bytearray
        ... and 215 other subclasses
    

    The initial list of subclasses to show would be restricted to public builtins: sorted((str(cls) for cls in object.__subclasses__() if not cls.__name__.startswith("_") and cls.__module__ == "builtins"), key=str.lower)

    If that gives less then four names, then the list of names would be expanded to subclasses with public names in public modules (i.e. neither __qualname__ nor __module__ starts with "_", neither of those contains "._", and qualname doesn't contain ".<locals>.").

    The full count of currently imported subclasses would ignore whether the subclass names were public or not.

    @robcliffe
    Copy link
    Mannequin Author

    robcliffe mannequin commented Jul 15, 2018

    On 14/07/2018 13:44, Nick Coghlan wrote:

    Nick Coghlan <ncoghlan@gmail.com> added the comment:

    Reviewing the builtins in 3.7, I get the following results for builtin objects that have defined subclasses immediately after interpreter startup:

    =============
    >>> for name, obj in vars(builtins).items():
    .. if isinstance(obj, type) and name in str(obj):
    .. subclasses = type(obj).__subclasses__(obj)
    .. if subclasses:
    .. print(f"{obj}: {len(subclasses)}")
    ..
    <class 'classmethod'>: 1
    <class 'dict'>: 1
    <class 'property'>: 1
    <class 'int'>: 1
    <class 'object'>: 132
    <class 'staticmethod'>: 1
    <class 'tuple'>: 16
    <class 'type'>: 1
    <class 'BaseException'>: 4
    <class 'Exception'>: 19
    <class 'ImportError'>: 2
    <class 'OSError'>: 13
    <class 'RuntimeError'>: 3
    <class 'NameError'>: 1
    <class 'SyntaxError'>: 1
    <class 'IndentationError'>: 1
    <class 'LookupError'>: 3
    <class 'ValueError'>: 2
    <class 'UnicodeError'>: 3
    <class 'ArithmeticError'>: 3
    <class 'SystemError'>: 1
    <class 'Warning'>: 10
    <class 'ConnectionError'>: 4
    =============

    So rather than special-casing exceptions or builtins in general, my inclination would be to include a section that lists up to 4 subclasses inline, and then adds a "... and NNN additional subclasses" trailing when there are more than 4 (or when there are less than 4 subclasses with public names, but additional private subclasses).

    My 2 cents:
        To use Exceptions optimally (e.g. to make the errors you trap
    neither too specific nor too general), you often need to know (and
    understand) the relevant part of the Exception hierarchy.  In particular
    you may know the name of an Exception that covers a particular use case,
    but not the names of its subclasses, one of which might be more
    appropriate.  Exceptions are exceptional* in that the "issubclass"
    relationship is vital to the way that they work.  So it is USEFUL to
    know ALL subclasses of a given Exception class (not just 4; in practice
    there won't be more than a dozen or two).  I have found myself in just
    this position; in fact it impelled me to adding a "show subclasses"
    feature to help(<anyExceptionClass>) in my then current version of
    Python 2.  (An alternative might be a handy way of showing the complete
    built-in Exception hierarchy.)

    I question whether it is really useful to know all subclasses of ANY
    class, or whether YAGNI.  I think that, for non-Exception classes,
    typically when you look at a class you may want to know its inheritance
    (to understand its functionality better), but it is rare that you will
    want to know what subclasses it has, if any.  No doubt there are
    exceptions* (perhaps you monkey-patch a class and want to know what
    subclasses might be affected).
    Regards
    Rob Cliffe

    • Pun not intended

    @eric-wieser
    Copy link
    Mannequin

    eric-wieser mannequin commented Jul 23, 2018

    I get the following results for builtin objects that have defined subclasses

    From that list, the only unhelpful ones with > 4 items, in my opinion, appear to be object, since that just tells you every type that exists, and tuple, because that lists every single namedtuple.

    So it is USEFUL to know ALL subclasses of a given Exception class

    I agree with this - most of the value here comes from showing the full set of exceptions. If we don't do that, we should probably point the user to calling cls.__subclasses__() so they can inspect the full list

    @CuriousLearner
    Copy link
    Member

    Hi,

    My perception with all the discussion and WIP patch is that we can ideally limit the no. of subclasses shown only for object, and not for any other class.

    From that list, the only unhelpful ones with > 4 items, in my opinion, appear to be object, since that just tells you every type that exists, and tuple, because that lists every single namedtuple.

    So it is USEFUL to know ALL subclasses of a given Exception class

    +1

    I agree with this - most of the value here comes from showing the full set of exceptions. If we don't do that, we should probably point the user to calling cls.__subclasses__() so they can inspect the full list

    I agree with this. I would propose to only limit to 4 classes for subclasses of object and for everything else displaying all the subclasses. We can optionally display the message to call cls.__subclasses() in the case when help(object) is called.

    How does it sound?

    @ncoghlan
    Copy link
    Contributor

    This is a UX problem folks: dynamic unbounded lists are inherently problematic in a line based information display.

    For methods on a class, there's the natural constraint that broad class APIs are a problem for a variety of reasons, but there's no such design constraint for exception hierarchies.

    In a Fedora Python 3.6 shell (with the Fedora abrt exception handling hook loaded), Exception has 31 subclasses, and OSError has 17.

    In a 3.6 shell venv (with no Fedora abrt hook), those numbers are 20 and 13.

    But if you import assorted Django modules then the number of Exception subclasses grows significantly, while the number of OSError subclasses grows slightly:

    >>> import django
    >>> len(list(pkgutil.iter_modules(django.__path__)))
    17
    >>> len(Exception.__subclasses__())
    76
    >>> len(OSError.__subclasses__())
    22

    Having "help(Exception)" list 50+ different Django exceptions is unlikely to be informative for anyone.

    In many ways, Python 2 was actually better in this regard, since you could just do "import exceptions; help(exceptions)" to get a nice overview of the full exception hierarchy using the standard logic for displaying.

    The closest Python 3 currently gets to that is "import builtins; help(builtins)", which is significantly harder to discover.

    @ncoghlan
    Copy link
    Contributor

    I think the way modules display this information may actually provide a solid precedent that addresses the dynamic state problem I'm concerned about: filter the subclass list to only include subclasses that come from the same module as the object being displayed, and use the existing showtree/getclasstree interfaces to display the nested hierarchy nicely (the same way modules do).

    Then the "... and NNN other subclasses in other modules" would only be appended when there was cross-module inheritance involved.

    @ncoghlan
    Copy link
    Contributor

    New changeset a323cdc by Nick Coghlan (Sanyam Khurana) in branch 'master':
    bpo-8525: help() on a type now shows builtin subclasses (GH-5066)
    a323cdc

    @ncoghlan
    Copy link
    Contributor

    I've merged the version that displays up to 4 builtin subclasses in a flat list when help() is called on a type. Thanks for the patch Sanyam, and for the comments and suggestions everyone else.

    While I'm closing out this feature request as implemented, if anyone's interested in pursuing the more sophisticated showtree/getclasstree approach that would better show position in the class hierarchy (both parents *and* children), feel free to file a new enhancement issue that refers back to this one.

    @serhiy-storchaka
    Copy link
    Member

    This caused a regression. See bpo-35614 for details.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    8 participants