classification
Title: Display exceptions' subclasses in help()
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: CuriousLearner Nosy List: CuriousLearner, Eric.Wieser, Rodolpho.Eckhardt, belopolsky, brian.curtin, eric.araujo, georg.brandl, henriquebastos, ncoghlan, robcliffe, ron_adam, serhiy.storchaka, terry.reedy, xtreak
Priority: normal Keywords: patch

Created on 2010-04-24 23:10 by robcliffe, last changed 2018-12-29 15:22 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
pydoc.py robcliffe, 2010-04-24 23:09 pydoc.py with enhancement to help()
help_8525.patch Rodolpho.Eckhardt, 2010-11-21 15:25 review
Pull Requests
URL Status Linked Edit
PR 5066 merged CuriousLearner, 2017-12-31 18:49
Messages (26)
msg104134 - (view) Author: Rob Cliffe (robcliffe) Date: 2010-04-24 23:09
help() on an exception class lists the method resolution order, which is in effect the class inheritance hierarchy.  E.g. help(ArithmeticError) lists ArithmeticError, StandardError, Exception, BaseException, __builtin__.object.  It struck me it would help to find my way around if it also listed the builtin SUBclasses (if any).  Something like:
    Built-in subclasses:
        FloatingPointError
        OverflowError
        ZeroDivisionError
In fact why not do it for any class, not just exceptions?
I attach a patched version of pydoc.py - tested but only on my PC which is running Python 2.5 under Windows XP.  I have added lines 1129-1148 to the docclass method of the TextDoc class (and flagged them # [RAC] ).
(I don't pretend to understand the magic where __builtins__ is a dictionary when pydoc.py is run but becomes a module later on.  Never mind - the patch works (I believe).)
For consistency, a similar patch would also have to be made to the docclass nethod of the HTMLDoc class (which outputs HTML rather than plain text).  I have not attempted this as I don't know how it is called and hence how to test any patch, but it should be straightforward for anyone with the know-how.
msg104656 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-04-30 19:10
2.5 is frozen. 2.6 and 3.1 are in bug-fix mode only, and 2.7 is in beta so new features are unlikely there. So I recommend doing a patch against 3.1 or even the 3.2 trunk. And of course, make sure this proposal makes sense for 3.x.
msg104657 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-04-30 19:22
Thanks for your work, Rob. To get reviews and comments, you’ll need to submit a patch (a diff) instead of the whole file. This makes it easier to see your changes than looking for a special comment :)

This short doc should contain all the information you’ll need: http://www.python.org/dev/patches

As Terry said, 2.6 and 3.1 only get bug fixes, 2.7 and 3.2 are in beta stage and don’t get new features, so you’ll want to port your changes to the py3k branch (the trunk for the 3.x series).

Regards
msg104658 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-04-30 19:24
Minor correction to the last comment: 3.2 is not in beta nor feature freeze.
msg121940 - (view) Author: Rodolpho Eckhardt (Rodolpho.Eckhardt) Date: 2010-11-21 15:25
Migrated the patch to the 3.x trunk and added three tests.
msg122044 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-22 00:26
Thank you.  I uploaded your patch to Rietveld and reviewed it: http://codereview.appspot.com/3169042/
msg122057 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-11-22 01:33
The following passes tests in elp_8525.patch, but is much simpler:

===================================================================
--- Lib/pydoc.py	(revision 86600)
+++ Lib/pydoc.py	(working copy)
@@ -1139,6 +1139,15 @@
                 push('    ' + makename(base))
             push('')
 
+        # List the built-in subclasses, if any:
+        subclasses = [cls.__name__ for cls in object.__subclasses__()
+                      if cls.__module__ == 'builtins']
+        if subclasses:
+            push("Built-in subclasses:")
+            for subclassname in sorted(subclasses):
+                push('    ' + subclassname)
+            push('')
+
         # Cute little class to pump out a horizontal rule between sections.
         class HorizontalRule:
             def __init__(self):
msg122115 - (view) Author: Rob Cliffe (robcliffe) Date: 2010-11-22 12:48
Thanks for your work.  Glad if I have made a contribution to Python, 
however small.
Rob Cliffe

On 22/11/2010 00:26, Éric Araujo wrote:
> Éric Araujo<merwok@netwok.org>  added the comment:
>
> Thank you.  I uploaded your patch to Rietveld and reviewed it: http://codereview.appspot.com/3169042/
>
> ----------
>
> _______________________________________
> Python tracker<report@bugs.python.org>
> <http://bugs.python.org/issue8525>
> _______________________________________
>
msg122116 - (view) Author: Rob Cliffe (robcliffe) Date: 2010-11-22 12:58
I would not be at all surprised if my patch could be simplified (in fact 
I'd be surprised if it couldn't).
However, I did try out your version on Python 2.5 specifically, and it 
did not work for me.
Trying it out on help(Exception), the relevant members of 
object.__subclasses__() viz.
<type 'exceptions.StandardError'>, <type 'exceptions.StopIteration'>  etc.
had a __module__attribute which equalled 'exceptions', not 'builtins'.
Best wishes
Rob Cliffe

On 22/11/2010 01:33, Alexander Belopolsky wrote:
> Alexander Belopolsky<belopolsky@users.sourceforge.net>  added the comment:
>
> The following passes tests in elp_8525.patch, but is much simpler:
>
> ===================================================================
> --- Lib/pydoc.py	(revision 86600)
> +++ Lib/pydoc.py	(working copy)
> @@ -1139,6 +1139,15 @@
>                   push('    ' + makename(base))
>               push('')
>
> +        # List the built-in subclasses, if any:
> +        subclasses = [cls.__name__ for cls in object.__subclasses__()
> +                      if cls.__module__ == 'builtins']
> +        if subclasses:
> +            push("Built-in subclasses:")
> +            for subclassname in sorted(subclasses):
> +                push('    ' + subclassname)
> +            push('')
> +
>           # Cute little class to pump out a horizontal rule between sections.
>           class HorizontalRule:
>               def __init__(self):
>
> ----------
> nosy: +belopolsky
>
> _______________________________________
> Python tracker<report@bugs.python.org>
> <http://bugs.python.org/issue8525>
> _______________________________________
>
msg122118 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-22 13:05
New features can only go into 3.2, so you have to test with an updated checkout of the Subversion branch named py3k.  See the link I have in a previous message.

(P.S. Would you be so kind as to edit quoted text out of your replies?  It’s unnecessary noise.  Thanks in advance.)
msg122193 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-11-23 05:34
The proposal is to display builtin subclasses as for example:

>>> help(ArithmeticError)
class ArithmeticError(Exception)
 |  Base class for arithmetic errors.
 |  
 |  Method resolution order:
 |      ArithmeticError
 |      Exception
 |      BaseException
 |      object
 |  
 |  Built-in subclasses:
 |      FloatingPointError
 |      OverflowError
 |      ZeroDivisionError

Note that this really affects only exceptions because no other builtin class has builtin subclasses.  (dict has subclasses in collections, but not in builtins.)

Exception hierarchy is presented in the reference manual at

http://docs.python.org/dev/library/exceptions.html?#exception-hierarchy

I wonder if rather than having MRO and subclasses sections, we should just present a portion of the exception hierarchy including the given exception, all its bases and direct subclasses:

object
 |
BaseException
 |
Exception
 |
*ArithmeticError*
 |
 +-- FloatingPointError
 +-- OverflowError
 +-- ZeroDivisionError
msg123167 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-03 02:56
Didn’t the first message ask for the feature to be extended to non-exceptions classes?  “Built-in” subclasses is a red herring, to me the feature is: display subclasses.  In the text output you can use the search feature of your pager to jump to a subclass, in the HTML output it would be a helpful link.
msg123169 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-12-03 03:02
On Thu, Dec 2, 2010 at 9:56 PM, Éric Araujo <report@bugs.python.org> wrote:
..
> Didn’t the first message ask for the feature to be extended to non-exceptions classes?  “Built-in”
> subclasses is a red herring, to me the feature is: display subclasses.  In the text output you can use
> the search feature of your pager to jump to a subclass, in the HTML output it would be a helpful link.

If we don't restrict to builtins, then the display will depend on what
modules are currently loaded and will be useless for command line use
unless all subclasses are defined in the same module.
msg123170 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-03 03:06
Yes, I imagine the feature to be useful to find subclasses defined in the same module.  Rob, am I misunderstanding your original request?
msg123228 - (view) Author: Rob Cliffe (robcliffe) Date: 2010-12-03 11:54
Originally I only had built-in classes in mind (with Exceptions being 
IMO the most obvious example of a built-in class hierarchy that it is 
useful to find your way around).  But if the idea can be extended to 
other classes, well, great.
Rob Cliffe
msg288243 - (view) Author: Sanyam Khurana (CuriousLearner) * (Python triager) Date: 2017-02-20 21:21
Hi,

It seems that it hasn't been worked upon from quite a long time and the patch would also need changes. May I work on this?
msg309296 - (view) Author: Sanyam Khurana (CuriousLearner) * (Python triager) Date: 2017-12-31 18:52
I've changed the version to 3.7. Not sure if this would need a backport since this is a new feature. So, I'll defer this call to a core developer. I've created a PR for the new feature.
msg321651 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-07-14 12:44
Reviewing the builtins in 3.7, I get the following results for builtin objects that have defined subclasses immediately after interpreter startup:

=============
>>> for name, obj in vars(builtins).items():
...     if isinstance(obj, type) and name in str(obj):
...         subclasses = type(obj).__subclasses__(obj)
...         if subclasses:
...             print(f"{obj}: {len(subclasses)}")
... 
<class 'classmethod'>: 1
<class 'dict'>: 1
<class 'property'>: 1
<class 'int'>: 1
<class 'object'>: 132
<class 'staticmethod'>: 1
<class 'tuple'>: 16
<class 'type'>: 1
<class 'BaseException'>: 4
<class 'Exception'>: 19
<class 'ImportError'>: 2
<class 'OSError'>: 13
<class 'RuntimeError'>: 3
<class 'NameError'>: 1
<class 'SyntaxError'>: 1
<class 'IndentationError'>: 1
<class 'LookupError'>: 3
<class 'ValueError'>: 2
<class 'UnicodeError'>: 3
<class 'ArithmeticError'>: 3
<class 'SystemError'>: 1
<class 'Warning'>: 10
<class 'ConnectionError'>: 4
=============

So rather than special-casing exceptions or builtins in general, my inclination would be to include a section that lists up to 4 subclasses inline, and then adds a "... and NNN additional subclasses" trailing when there are more than 4 (or when there are less than 4 subclasses with public names, but additional private subclasses).

So in a class like "int", for example, we'd see:

    Currently imported subclasses:
        bool

While in a class like OSError we'd see:

    Currently imported subclasses:
        BlockingIOError
        ChildProcessError
        ConnectionError
        FileExistsError
        ... and 9 other subclasses

And in "help(object)" we'd see something like:

    Currently imported subclasses:
        async_generator
        BaseException
        builtin_function_or_method
        bytearray
        ... and 215 other subclasses


The initial list of subclasses to show would be restricted to public builtins: `sorted((str(cls) for cls in object.__subclasses__() if not cls.__name__.startswith("_") and cls.__module__ == "builtins"), key=str.lower)`

If that gives less then four names, then the list of names would be expanded to subclasses with public names in public modules (i.e. neither __qualname__ nor __module__ starts with "_", neither of those contains "._", and qualname doesn't contain ".<locals>.").

The full count of currently imported subclasses would ignore whether the subclass names were public or not.
msg321675 - (view) Author: Rob Cliffe (robcliffe) Date: 2018-07-15 03:16
On 14/07/2018 13:44, Nick Coghlan wrote:
> Nick Coghlan <ncoghlan@gmail.com> added the comment:
>
> Reviewing the builtins in 3.7, I get the following results for builtin objects that have defined subclasses immediately after interpreter startup:
>
> =============
>>>> for name, obj in vars(builtins).items():
> ..     if isinstance(obj, type) and name in str(obj):
> ..         subclasses = type(obj).__subclasses__(obj)
> ..         if subclasses:
> ..             print(f"{obj}: {len(subclasses)}")
> ..
> <class 'classmethod'>: 1
> <class 'dict'>: 1
> <class 'property'>: 1
> <class 'int'>: 1
> <class 'object'>: 132
> <class 'staticmethod'>: 1
> <class 'tuple'>: 16
> <class 'type'>: 1
> <class 'BaseException'>: 4
> <class 'Exception'>: 19
> <class 'ImportError'>: 2
> <class 'OSError'>: 13
> <class 'RuntimeError'>: 3
> <class 'NameError'>: 1
> <class 'SyntaxError'>: 1
> <class 'IndentationError'>: 1
> <class 'LookupError'>: 3
> <class 'ValueError'>: 2
> <class 'UnicodeError'>: 3
> <class 'ArithmeticError'>: 3
> <class 'SystemError'>: 1
> <class 'Warning'>: 10
> <class 'ConnectionError'>: 4
> =============
>
> So rather than special-casing exceptions or builtins in general, my inclination would be to include a section that lists up to 4 subclasses inline, and then adds a "... and NNN additional subclasses" trailing when there are more than 4 (or when there are less than 4 subclasses with public names, but additional private subclasses).
>
>
My 2 cents:
     To use Exceptions optimally (e.g. to make the errors you trap 
neither too specific nor too general), you often need to know (and 
understand) the relevant part of the Exception hierarchy.  In particular 
you may know the name of an Exception that covers a particular use case, 
but not the names of its subclasses, one of which might be more 
appropriate.  Exceptions are exceptional* in that the "issubclass" 
relationship is vital to the way that they work.  So it is USEFUL to 
know ALL subclasses of a given Exception class (not just 4; in practice 
there won't be more than a dozen or two).  I have found myself in just 
this position; in fact it impelled me to adding a "show subclasses" 
feature to help(<anyExceptionClass>) in my then current version of 
Python 2.  (An alternative might be a handy way of showing the complete 
built-in Exception hierarchy.)

I question whether it is really useful to know all subclasses of ANY 
class, or whether YAGNI.  I think that, for non-Exception classes, 
typically when you look at a class you may want to know its inheritance 
(to understand its functionality better), but it is rare that you will 
want to know what subclasses it has, if any.  No doubt there are 
exceptions* (perhaps you monkey-patch a class and want to know what 
subclasses might be affected).
Regards
Rob Cliffe

* Pun not intended
msg322226 - (view) Author: Eric Wieser (Eric.Wieser) * Date: 2018-07-23 15:49
> I get the following results for builtin objects that have defined subclasses

From that list, the only unhelpful ones with > 4 items, in my opinion, appear to be `object`, since that just tells you every type that exists, and `tuple`, because that lists every single namedtuple.

> So it is USEFUL to know ALL subclasses of a given Exception class

I agree with this - most of the value here comes from showing the full set of exceptions. If we don't do that, we should probably point the user to calling `cls.__subclasses__()` so they can inspect the full list
msg322240 - (view) Author: Sanyam Khurana (CuriousLearner) * (Python triager) Date: 2018-07-23 18:42
Hi,

My perception with all the discussion and WIP patch is that we can ideally limit the no. of subclasses shown only for object, and not for any other class.

>From that list, the only unhelpful ones with > 4 items, in my opinion, appear to be `object`, since that just tells you every type that exists, and `tuple`, because that lists every single namedtuple.

> So it is USEFUL to know ALL subclasses of a given Exception class

+1

> I agree with this - most of the value here comes from showing the full set of exceptions. If we don't do that, we should probably point the user to calling `cls.__subclasses__()` so they can inspect the full list

I agree with this. I would propose to only limit to 4 classes for subclasses of `object` and for everything else displaying all the subclasses. We can optionally display the message to call `cls.__subclasses()` in the case when `help(object)` is called.

How does it sound?
msg322287 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-07-24 10:32
This is a UX problem folks: dynamic unbounded lists are inherently problematic in a line based information display.

For methods on a class, there's the natural constraint that broad class APIs are a problem for a variety of reasons, but there's no such design constraint for exception hierarchies.

In a Fedora Python 3.6 shell (with the Fedora abrt exception handling hook loaded), Exception has 31 subclasses, and OSError has 17.

In a 3.6 shell venv (with no Fedora abrt hook), those numbers are 20 and 13. 

But if you import assorted Django modules then the number of Exception subclasses grows significantly, while the number of OSError subclasses grows slightly:

>>> import django
>>> len(list(pkgutil.iter_modules(django.__path__)))
17
>>> len(Exception.__subclasses__())
76
>>> len(OSError.__subclasses__())
22

Having "help(Exception)" list 50+ different Django exceptions is unlikely to be informative for anyone.


In many ways, Python 2 was actually better in this regard, since you could just do "import exceptions; help(exceptions)" to get a nice overview of the full exception hierarchy using the standard logic for displaying.


The closest Python 3 currently gets to that is "import builtins; help(builtins)", which is significantly harder to discover.
msg322288 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-07-24 10:47
I think the way modules display this information may actually provide a solid precedent that addresses the dynamic state problem I'm concerned about: filter the subclass list to only include subclasses that come from the same module as the object being displayed, and use the existing showtree/getclasstree interfaces to display the nested hierarchy nicely (the same way modules do).

Then the "... and NNN other subclasses in other modules" would only be appended when there was cross-module inheritance involved.
msg328194 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-10-21 07:22
New changeset a323cdcb33c8c856e5668acfb2c67ab5198672c4 by Nick Coghlan (Sanyam Khurana) in branch 'master':
bpo-8525: help() on a type now shows builtin subclasses (GH-5066)
https://github.com/python/cpython/commit/a323cdcb33c8c856e5668acfb2c67ab5198672c4
msg328195 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-10-21 07:27
I've merged the version that displays up to 4 builtin subclasses in a flat list when help() is called on a type. Thanks for the patch Sanyam, and for the comments and suggestions everyone else.

While I'm closing out this feature request as implemented, if anyone's interested in pursuing the more sophisticated showtree/getclasstree approach that would better show position in the class hierarchy (both parents *and* children), feel free to file a new enhancement issue that refers back to this one.
msg332721 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-12-29 15:22
This caused a regression. See issue35614 for details.
History
Date User Action Args
2018-12-29 15:22:31serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg332721
2018-10-21 07:27:06ncoghlansetstatus: open -> closed
resolution: fixed
messages: + msg328195

stage: patch review -> resolved
2018-10-21 07:22:06ncoghlansetmessages: + msg328194
2018-07-24 12:23:50xtreaksetnosy: + xtreak
2018-07-24 10:47:46ncoghlansetmessages: + msg322288
2018-07-24 10:32:54ncoghlansetmessages: + msg322287
2018-07-23 18:42:31CuriousLearnersetmessages: + msg322240
2018-07-23 15:49:52Eric.Wiesersetnosy: + Eric.Wieser
messages: + msg322226
2018-07-15 03:16:28robcliffesetmessages: + msg321675
2018-07-14 12:44:18ncoghlansetnosy: + ncoghlan
messages: + msg321651
2018-02-08 20:14:51ppperrysettitle: Display exception's subclasses in help() -> Display exceptions' subclasses in help()
2018-02-08 19:24:19CuriousLearnersetassignee: CuriousLearner
versions: + Python 3.8, - Python 3.7
2017-12-31 18:52:23CuriousLearnersetmessages: + msg309296
versions: + Python 3.7, - Python 3.2
2017-12-31 18:49:24CuriousLearnersetstage: needs patch -> patch review
pull_requests: + pull_request4940
2017-02-20 21:21:58CuriousLearnersetnosy: + CuriousLearner
messages: + msg288243
2012-10-06 11:15:52georg.brandlsetassignee: georg.brandl -> (no value)
2010-12-03 11:54:25robcliffesetmessages: + msg123228
2010-12-03 03:06:31eric.araujosetmessages: + msg123170
2010-12-03 03:02:16belopolskysetmessages: + msg123169
2010-12-03 02:56:38eric.araujosetmessages: + msg123167
2010-11-23 05:34:26belopolskysetmessages: + msg122193
title: Small enhancement to help() -> Display exception's subclasses in help()
2010-11-22 13:05:48eric.araujosetmessages: + msg122118
2010-11-22 12:58:56robcliffesetmessages: + msg122116
2010-11-22 12:48:20robcliffesetmessages: + msg122115
2010-11-22 01:33:35belopolskysetnosy: + belopolsky
messages: + msg122057
2010-11-22 00:26:21eric.araujosetmessages: + msg122044
2010-11-21 15:25:07Rodolpho.Eckhardtsetfiles: + help_8525.patch

nosy: + henriquebastos, Rodolpho.Eckhardt
messages: + msg121940

keywords: + patch
2010-04-30 19:24:06brian.curtinsetnosy: + brian.curtin
messages: + msg104658
2010-04-30 19:22:53eric.araujosetnosy: + eric.araujo
messages: + msg104657

components: + Library (Lib)
stage: needs patch
2010-04-30 19:10:24terry.reedysetnosy: + terry.reedy

messages: + msg104656
versions: + Python 3.2, - Python 2.5
2010-04-25 06:16:02georg.brandlsetassignee: georg.brandl

nosy: + georg.brandl
2010-04-25 03:15:26ron_adamsetnosy: + ron_adam
2010-04-24 23:10:00robcliffecreate