classification
Title: pdb error when stepping through generator
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: Nosy List: Ricyteach, pitrou, serhiy.storchaka, xdegaye, yselivanov
Priority: normal Keywords:

Created on 2018-04-22 03:03 by Ricyteach, last changed 2018-04-29 18:01 by Ricyteach. This issue is now closed.

Files
File name Uploaded Description Edit
demo.py Ricyteach, 2018-04-22 03:03 demo of error
Messages (6)
msg315586 - (view) Author: Rick Teachey (Ricyteach) * Date: 2018-04-22 03:03
Hello: 

The attached python file runs a pdb interactive session for a generator. It produces an error in Python 3.7 Beta 3, but no error in 3.6.

I searched for this issue and there do seem to be things that are related to it already, but I haven't investigated the cause of this issue and am not acquainted with the details of the pdb module, so I was unable to determine whether previous reports discuss this same problem.
msg315587 - (view) Author: Rick Teachey (Ricyteach) * Date: 2018-04-22 03:05
The interactive session result is below:

GeneratorExit

Exception ignored in: <async_generator object _ag at 0x0000019E069AAD90>
Traceback (most recent call last):
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\types.py", line 27, in _ag
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\bdb.py", line 90, in trace_dispatch
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\bdb.py", line 128, in dispatch_call
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\bdb.py", line 250, in break_anywhere
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\bdb.py", line 49, in canonic
AttributeError: 'NoneType' object has no attribute 'abspath'
msg315588 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-22 05:45
Thank you for your report Rick. But I get the same error in 3.6.

An AttributeError is raised if the debugger runs at the interpreter shutdown stage (sys.is_finalizing() returns True). sys.path was set to None at the beginning of modules cleanup. The debugger traces the _ag() asynchronous generator in types.py. You can see some details when run Python with option -v.

$ ./python -v ../demo.py
...
import 'pdb' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5b45debae8>
# extension module 'readline' loaded from '/home/serhiy/py/cpython/build/lib.linux-x86_64-3.8-pydebug/readline.cpython-38dm-x86_64-linux-gnu.so'
# extension module 'readline' executed from '/home/serhiy/py/cpython/build/lib.linux-x86_64-3.8-pydebug/readline.cpython-38dm-x86_64-linux-gnu.so'
import 'readline' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7f5b45d94608>
GeneratorExit

# clear builtins._
# clear sys.path
# clear sys.argv
...
# cleanup[3] wiping re
# destroy _locale
# destroy sre_compile
# destroy copyreg
# cleanup[3] wiping enum
# cleanup[3] wiping types
Exception ignored in: <async_generator object _ag at 0x7f6de03f0eb8>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/types.py", line 27, in _ag
  File "/home/serhiy/py/cpython/Lib/bdb.py", line 90, in trace_dispatch
  File "/home/serhiy/py/cpython/Lib/bdb.py", line 128, in dispatch_call
  File "/home/serhiy/py/cpython/Lib/bdb.py", line 250, in break_anywhere
  File "/home/serhiy/py/cpython/Lib/bdb.py", line 49, in canonic
AttributeError: 'NoneType' object has no attribute 'abspath'

Adding "del _ag" in types.py fixes this issue. But there are open questions.

Is it good that the debugger is enabled at the shutdown stage?

Why the code of the never used asynchronous generator is executed at all?
msg315589 - (view) Author: Rick Teachey (Ricyteach) * Date: 2018-04-22 06:17
I am on Anaconda 4.5.1 on Windows 10 (Python 3.6.5). I have confirmed that I DO get the error on another machine with the same version installed, so the lack of an error seems specific to my current machine. I do not know what could be causing this; it is very odd.
msg315893 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2018-04-29 12:58
This is a duplicate of issue 13044.

> Why the code of the never used asynchronous generator is executed at all?

The destructor of the asynchronous generator is called during python finalization while the trace is still active. Here is the backtrace where python is stopped just upon entering the debugger and before the exception is raised by bdb:

#0  call_trace_protected (func=0x5555556c0a5b <trace_trampoline>, 
    obj=<method at remote 0x7ffff6f46c98>, tstate=tstate@entry=0x555555af5eb0, 
    frame=frame@entry=Frame 0x7ffff6e63218, for file /home/xavier/src/python/master/Lib/types.py, line 27, in _ag (), what=what@entry=0, arg=None) at Python/ceval.c:4228
#1  0x00005555556757f9 in _PyEval_EvalFrameDefault (
    f=Frame 0x7ffff6e63218, for file /home/xavier/src/python/master/Lib/types.py, line 27, in _ag (), throwflag=1) at Python/ceval.c:843
#2  0x000055555567445d in PyEval_EvalFrameEx (
    f=f@entry=Frame 0x7ffff6e63218, for file /home/xavier/src/python/master/Lib/types.py, line 27, in _ag (), throwflag=throwflag@entry=1) at Python/ceval.c:536
#3  0x000055555575c0d2 in gen_send_ex (gen=gen@entry=0x7ffff6e27d48, arg=<optimized out>, 
    exc=exc@entry=1, closing=closing@entry=1) at Objects/genobject.c:221
#4  0x000055555575c66b in gen_close (gen=gen@entry=0x7ffff6e27d48, args=args@entry=0x0)
    at Objects/genobject.c:371
#5  0x000055555575c7f1 in _PyGen_Finalize (self=<async_generator at remote 0x7ffff6e27d48>)
    at Objects/genobject.c:84
#6  0x00005555555f7c7d in PyObject_CallFinalizer (
    self=self@entry=<async_generator at remote 0x7ffff6e27d48>) at Objects/object.c:286
#7  0x00005555555f89c0 in PyObject_CallFinalizerFromDealloc (
    self=self@entry=<async_generator at remote 0x7ffff6e27d48>) at Objects/object.c:303
#8  0x000055555575ba27 in gen_dealloc (gen=gen@entry=0x7ffff6e27d48) at Objects/genobject.c:127
#9  0x00005555555f8a7e in _Py_Dealloc (op=<async_generator at remote 0x7ffff6e27d48>)
    at Objects/object.c:1933
#10 0x00005555555ea25b in insertdict (mp=mp@entry=0x7ffff6e53d00, key=key@entry='_ag', 
    hash=-1842969574970988536, value=value@entry=None) at Objects/dictobject.c:1067
#11 0x00005555555eb8a0 in PyDict_SetItem (
    op=op@entry=Python Exception <class 'gdb.error'> Attempt to extract a component of a value that is not a struct/class/union.: 
, key='_ag', value=None) at Objects/dictobject.c:1454
#12 0x00005555555f764f in _PyModule_ClearDict (
    d=Python Exception <class 'gdb.error'> Attempt to extract a component of a value that is not a struct/class/union.: 
) at Objects/moduleobject.c:606
#13 0x00005555555f793f in _PyModule_Clear (m=m@entry=<module at remote 0x7ffff6e57758>)
    at Objects/moduleobject.c:577
#14 0x000055555569debe in PyImport_Cleanup () at Python/import.c:544
#15 0x00005555556ac463 in Py_FinalizeEx () at Python/pylifecycle.c:1094
#16 0x00005555555b2136 in pymain_main (pymain=pymain@entry=0x7fffffffe420) at Modules/main.c:2664
#17 0x00005555555b22c3 in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>)
    at Modules/main.c:2697
#18 0x00005555555ad403 in main (argc=<optimized out>, argv=<optimized out>)
    at ./Programs/python.c:15
msg315903 - (view) Author: Rick Teachey (Ricyteach) * Date: 2018-04-29 18:01
Closed as duplicate of issue 13044.
History
Date User Action Args
2018-04-29 18:01:22Ricyteachsetstatus: open -> closed
resolution: duplicate
messages: + msg315903

stage: resolved
2018-04-29 12:58:52xdegayesetnosy: + xdegaye
messages: + msg315893
2018-04-22 06:17:43Ricyteachsetmessages: + msg315589
2018-04-22 05:45:39serhiy.storchakasetnosy: + yselivanov, serhiy.storchaka, pitrou

messages: + msg315588
versions: + Python 3.6, Python 3.8
2018-04-22 04:45:03Ricyteachsettype: behavior
components: + Library (Lib)
2018-04-22 03:05:51Ricyteachsetmessages: + msg315587
2018-04-22 03:03:53Ricyteachcreate