If pdb.set_trace() is the last statement in the first code snippet, variable 'err' is (wrongly?) excluded from locals(). Adding any code after the pdb.set_trace() statement makes 'err' available in locals.

In [2]: try:
   ...:     raise ValueError("I am ValueError")
   ...: except ValueError as err:
   ...:     print("err" in locals())
   ...:     import pdb; pdb.set_trace()
   ...:
True
--Return--
> <ipython-input-2-9d1238ac9b7a>(5)<module>()->None
-> import pdb; pdb.set_trace()
(Pdb) print("err" in locals())
False          <-------------- BUG??
(Pdb) c

In [3]: try:
   ...:     raise ValueError("I am ValueError")
   ...: except ValueError as err:
   ...:     print("err" in locals())
   ...:     import pdb; pdb.set_trace()
   ...:     import os # Dummy code - makes variable 'err' available inside the debugger.
   ...:
True
> <ipython-input-3-3cefa7fe6ddd>(6)<module>()->None
-> import os # Dummy code - makes variable err available inside debugger
(Pdb) print("err" in locals())
True

In [4]: sys.version_info
Out[4]: sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)

FTR, the variable 'err' is available in both cases in case of Python 2.7. Also, this happens with ipdb as well.

Please note that I am aware that I need to assign the variable 'err' to some variable inside the except block to access it outside the except block. However, the pdb statement is still inside the except block in both cases.

Can confirm also on 3.6, but this seems to be something particular to set_trace, when pdb is entered via post_mortem, the variable is available in locals().

This is not a bug.
On the first pdb session from the OP:

True
--Return--
> <ipython-input-2-9d1238ac9b7a>(5)<module>()->None
-> import pdb; pdb.set_trace()
(Pdb) print("err" in locals())
False          <-------------- BUG??

Pdb has stopped on a '--Return--' line event, so we are already outside the scope of the except clause and the 'err' local variable cannot be accessed outside the except block. The following code snippet shows that pdb stops at the line following the set_trace() call:

==================
def foo():
    try:
        1/0
    except Exception as err:
        import pdb; pdb.set_trace()
    x= 1

foo()
==================

And the corresponding pdb session:

$ python foo.py
> /path/to/foo.py(6)foo()
-> x= 1
(Pdb) err
*** NameError: name 'err' is not defined
(Pdb)

https://docs.python.org/3/reference/compound_stmts.html#the-try-statement says

=============
except E as N:
    foo
=============

is converted to the following statement:

=============
except E as N:
    try:
        foo
    finally:
        del N
=============

In the examples in this thread, foo is 'import pdb; pdb.set_trace()'. So, I was expecting 'del N' to be executed only after executing foo. This implies that err should have been available in the scope.

Further, isn't the code execution supposed to be stopped at the set_trace() line and not on the next line. Should it at least respect the scope (indentation) and not execute the '--Return--' line event until the user actually executes the next line (x=1 in your example)?

@Saim wrote
> https://docs.python.org/3/reference/compound_stmts.html#the-try-statement says

This is correct, the disassembly of foo.py below demonstrates that point: BEGIN_FINALLY at bytecode index 36 starts the implementation of the 'finally' clause that deletes 'err'. But pdb is a line debugger, not a debugger at the bytecode level, so when 'foo = 1' is replaced by 'import pdb; pdb.set_trace()', the first line where pdb may stop is at line 6 and since foo() last line is line 5, pdb stops at the 'return' event for this function.

=======   foo.py   ========
def foo():
    try:
        1/0
    except Exception as err:
        foo = 1

import dis
dis.dis(foo)
===========================
  2           0 SETUP_FINALLY           12 (to 14)

  3           2 LOAD_CONST               1 (1)
              4 LOAD_CONST               2 (0)
              6 BINARY_TRUE_DIVIDE
              8 POP_TOP
             10 POP_BLOCK
             12 JUMP_FORWARD            38 (to 52)

  4     >>   14 DUP_TOP
             16 LOAD_GLOBAL              0 (Exception)
             18 COMPARE_OP              10 (exception match)
             20 POP_JUMP_IF_FALSE       50
             22 POP_TOP
             24 STORE_FAST               0 (err)
             26 POP_TOP
             28 SETUP_FINALLY            8 (to 38)

  5          30 LOAD_CONST               1 (1)
             32 STORE_FAST               1 (foo)
             34 POP_BLOCK
             36 BEGIN_FINALLY
        >>   38 LOAD_CONST               0 (None)
             40 STORE_FAST               0 (err)
             42 DELETE_FAST              0 (err)
             44 END_FINALLY
             46 POP_EXCEPT
             48 JUMP_FORWARD             2 (to 52)
        >>   50 END_FINALLY
        >>   52 LOAD_CONST               0 (None)
             54 RETURN_VALUE
===========================

The *** Tracing *** section [1] of Objects/lnotab_notes.txt in the cpython repository provides detailed explanations for when the line trace function is called.

[1] https://github.com/python/cpython/blob/9d7b2c0909b78800d1376fd696f73824ea680463/Objects/lnotab_notes.txt#L63

This is pretty unintuitive from a user's stand point. Now, I need to *inconveniently* put some dummy code after the ser_trace call every time I want to access the exception inside the except block.

Also, this is a change in behavior from Python 2.7. Is this documented somewhere?

Saim, a .post_mortem could be used instead. As I noted, it works just fine.

Thanks, SilentGhost! However, should we try to fix set_trace as well to avoid hassles to other users?

I cannot imagine that the fix would be straightforward or that there is much use of this particular pattern. Perhaps, a note in the docs suggesting post_mortem() for except clauses over set_trace() would be more appropriate.

> Now, I need to *inconveniently* put some dummy code after the ser_trace call every time I want to access the exception inside the except block

Knowing that pdb stops just before the interpreter executes the line after pdb.set_trace(), what prevents you from writing the pdb.set_trace() statement before the last line of the except clause ?

> Knowing that pdb stops just before the interpreter executes the line after pdb.set_trace(), what prevents you from writing the pdb.set_trace() statement before the last line of the except clause ?

Of course, that can be done. The issue here is to get the correct and expected behavior from pdb/ipdb.

On a similar note, users might want to put just the set_trace() call and no other code in the block while writing the code or during debugging the exception caught.