classification
Title: Exception ABC doesn't work in Python 3 (but does in Python 2.7)
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Allow catching virtual subclasses in except clauses
View: 12029
Assigned To: Nosy List: eryksun, josh.r, merchise, ncoghlan, r.david.murray, rhettinger
Priority: normal Keywords:

Created on 2015-10-20 19:32 by merchise, last changed 2015-10-21 02:32 by eryksun. This issue is now closed.

Files
File name Uploaded Description Edit
tests.py merchise, 2015-10-20 19:32 A test command to reproduce the error.
Messages (7)
msg253246 - (view) Author: Medardo Rodriguez (merchise) Date: 2015-10-20 19:32
When I run the next code in all my installed versions of Python 3 (and the equivalent code for PyPy and CPython 2.7):

- Runs OK in CPython 2.7 equivalent code (using `ABCMeta` as meta-class).

- Fails in PyPy and all versions of Python 3.
msg253253 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-10-20 21:10
What are you trying to accomplish that needs an abstract base class?

APIError is not an Exception class, it is a meta-class, and thus it does not match the exception because it is not a *base* class of DriverError (it is DiverError's *meta* class).

Now, it is an interesting question whether we want the try/except statement's behavior to match issubclass, since intuitively we think the except in try except is doing the equivalent of issubclass (it isn't).

I don't know why this works in python2, but the class/meta-class machinery did undergo some significant changes.  So, it is possible that this is in fact a bug (ie: an unintentional change in semantics).  If so, it is also a doc bug, since the try statement docs clear say the expression must evaluate to a base class, and the virtual class (abc) docs *only* mention issubclass and isinstance.
msg253258 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-10-20 21:41
Thinking about this some more my explanation isn't quite clear (because my thinking wasn't quite clear): the problem isn't that APIError isn't *capable* of being a base class; I was wrong when I said it is itself a metaclass.  It is a class that has ABCMeta *as* its metaclass.  The problem is what I said in the second part: that APIError isn't a base class *of DriverError*.  The operant issue is that 'except' isn't doing issubclass, it is looking at the base class list of DriverError, and APIError isn't in it.

I'd have to see your 2.7 code to figure out why that version "works" :)
msg253268 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-10-20 23:27
In Python 2, PyErr_GivenExceptionMatches [1] calls PyObject_IsSubclass. To handle calling __subclasscheck__ in this case, 2.7 (but not 2.6) temporarily increases the recursion limit by 5. For example:

    class CMeta(type):
        def __subclasscheck__(self, other):
            import sys
            print 'recursion limit: %d' % sys.getrecursionlimit()
            frame = sys._getframe(1)
            n = 0
            while frame:
                n += 1
                frame = frame.f_back
            print 'frame: %d' % n
            return True

    class C(Exception):
        __metaclass__ = CMeta

    def f():
        try:
            f()
        except C:
            pass

    >>> sys.getrecursionlimit()
    1000
    >>> f()
    recursion limit: 1005
    frame: 1000
    >>> sys.getrecursionlimit()
    1000

If the recursion limit weren't temporarily increased, then trying to call __subclasscheck__ in the above case would raise another RuntimeError.

In Python 3, PyErr_GivenExceptionMatches [2] instead calls PyType_IsSubtype. See issue 2534. In that issue Antoine's reason for the change is that "otherwise there are some nasty issues with recursion checking". 

[1]: https://hg.python.org/cpython/file/v2.7.10/Python/errors.c#l84
[2]: https://hg.python.org/cpython/file/v3.5.0/Python/errors.c#l166
msg253269 - (view) Author: Medardo Rodriguez (merchise) Date: 2015-10-21 00:19
Thanks ~eryksun, you clarified me in where the error is located and possible patches for it.

We are programming a framework where is very important to do mappings between (1) the API and (2) the drivers layers.

We try to make our code available to both branches of Python (2 and 3).  This case is complex because a logical exception type definition (like "resource not found") is implemented different by disparate API consumer applications (these implementing the drivers), for example `httplib`, `couchdb` or any specific database management system.

So, the API layer have to find a way to uniform disparate exception "fauna" in its definitions at a logical level.

ABCs are the nice solution, only have to work as its conceptual definition said, including exceptions, in all Python versions.
msg253271 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2015-10-21 01:48
This looks like a duplicate of #12029. The delays on addressing that bug seem to indicate that initially there was a fairly substantial performance cost for all exception handling paths when ABC friendly checking was performed, but that was at least partially due to a bug in those code paths that didn't bypass the ABC machinery in the simple subclass case.

That said, I'm not sure it wouldn't cause other performance regressions (I made a comment to that effect on the original bug).
msg253273 - (view) Author: Medardo Rodriguez (merchise) Date: 2015-10-21 02:26
Josh Rosenberg, you are right, it's a duplication. Sorry I didn't find the issue #12029 before.
History
Date User Action Args
2015-10-21 02:32:09eryksunsetstatus: open -> closed
superseder: Allow catching virtual subclasses in except clauses
resolution: duplicate
stage: resolved
2015-10-21 02:26:42merchisesetmessages: + msg253273
2015-10-21 01:48:33josh.rsetnosy: + josh.r
messages: + msg253271
2015-10-21 00:19:40merchisesetmessages: + msg253269
2015-10-20 23:27:31eryksunsetversions: + Python 3.5, Python 3.6, - Python 3.2, Python 3.3
nosy: + eryksun

messages: + msg253268

components: + Interpreter Core
2015-10-20 21:41:38r.david.murraysetmessages: + msg253258
2015-10-20 21:10:16r.david.murraysetnosy: + rhettinger, r.david.murray, ncoghlan
messages: + msg253253
2015-10-20 19:32:31merchisecreate