classification
Title: IDLE debugger: failure stepping through module loading
Type: behavior Stage: test needed
Components: IDLE Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: brett.cannon, jcdlr, om364@, ppperry, terry.reedy
Priority: normal Keywords:

Created on 2018-03-13 05:02 by jcdlr, last changed 2018-10-01 02:09 by ppperry.

Files
File name Uploaded Description Edit
PositionalList.py om364@, 2018-04-11 22:50 Linked List module user created
Messages (7)
msg313721 - (view) Author: Joshua De La Rosa (jcdlr) Date: 2018-03-13 05:02
Taking my first coding class, so I don't know much about coding or python in general, but I  ran into a problem when using the Debugger function for a homework assignment that neither I nor my professor could make sense of. My program executes successfully without running the Debugger or, in the case that I am running the Debugger, it only raises an error when I "Step" through the imported module that I implemented in another program, rather than just hitting "Go". The error it reports is:
AttributeError: '_ModuleLock' object has no attribute 'name'

Not sure which file to submit, since the I created my own module that is used in the program that raises the error when I step through it with the Debugger mode on.
msg313744 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-03-13 12:26
I cannot do anything from this bare description, as I cannot think of any reason why stepping should introduce an error.  It sounds like there are two files involved.  Try to reduce them to the minimum needed to reproduce the issue and then upload.
msg315203 - (view) Author: (om364@) Date: 2018-04-11 22:50
Traceback (most recent call last):
  File "...\PositionalList.py", line 1, in <module>
    from _DoublyLinkedBase import _DoublyLinkedBase
  File "<frozen importlib._bootstrap>", line 968, in _find_and_load
  File "<frozen importlib._bootstrap>", line 148, in __enter__
  File "<frozen importlib._bootstrap>", line 174, in _get_module_lock
  File "<frozen importlib._bootstrap>", line 59, in __init__
  File "<frozen importlib._bootstrap>", line 59, in __init__
  File "...\Python\Python36\lib\bdb.py", line 48, in trace_dispatch
    return self.dispatch_line(frame)
  File "...\Python\Python36\lib\bdb.py", line 66, in dispatch_line
    self.user_line(frame)
  File "...\Python\Python36\lib\idlelib\debugger.py", line 24, in user_line
    self.gui.interaction(message, frame)
AttributeError: '_ModuleLock' object has no attribute 'name'

The file works correctly in PowerShell, but in IDLE debbuger there is a error.
msg315208 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-04-11 23:56
We use 'crash' for, on Window, a process stopping either with no explanation or a 'Your process has stopped box' from Windows.

In any case, PositionalList.py will not run without _DoublyLinkedBase.py, which you did not upload.

The traceback has
  File "...\Python\Python36\lib\bdb.py", line 48, in trace_dispatch
    return self.dispatch_line(frame)

Since sometime last summer, that code line is line 51, indicating that you are using an old release of 3.6, probably 3.6.2 or earlier.  The current bugfix release is 3.6.5.  If possible, upgrade to 3.6.5 and retest.

The traceback does not make much sense to me either.  Neither of the IDLE methods userline and interaction obviously access a .name attribute of anything.  The '_ModuleLock' object is created by importlib._bootstrap.  Searching all issues for '_ModuleLock' got 11 hits.  None are obviously about import and tracing.

To determine whether the problem has anything to do with IDLE, single-step debugging should be repeated with pdb, which is Python's text debugger, and also based on bdb.

Brett, can you tell anything from the multiple importlib._bootstrap lines in the traceback?
msg315228 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-04-12 16:36
Without knowing the exact Python version it's hard to tell as line 59 changed between Python 3.5 and where Python 3.6 is now (FYI, the line as it currently sits in Python 3.6 is https://github.com/python/cpython/blob/e98e3385f2acfc6d98f70f8e66c96b752d003b8f/Lib/importlib/_bootstrap.py#L59). But the double-reporting of the same line is a bit odd.

The best I can think of is that IDLE is requesting the __repr__() of `self` while still executing `__init__()` but before `self.name` is set, triggering an AttributeError (although those lines don't exactly line up with that). Otherwise looking at the code for _ModuleLock suggests this really shouldn't happen as `self.name` is set in `__init__()` and there's no use of `del` or `delattr()` that would cause this.
msg326179 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-09-23 20:32
The problem is not limited to user modules.  In duplicate issue #34609 (not closed), the same traceback was obtained with unittest.  I also got the same with asyncio.  There will not be a problem if the module is already loaded in sys.modules, so that importlib is not invoked.

The traceback is obscured by the fact that the executed importlib is frozen, leaving it traceable, but the code not directly available.  Hence the code is omitted from the debugger display and traceback.  However, the line numbers can be used to find the code within Lib/importlib._bootstrap.py.  Using current master (updated last night), the functions and lines executed by stepping with import unittest are (as expected when reading the code, up to the unexpected exception):

_find_and_load: 980
ModuleLock.__init__: 144, 145
ModuleLock.__enter__: 148
_get_module_lock: 163-168, 170-171, 174: lock = _ModuleLock(name)
_ModuleLock.__init__: 59: self.lock = _thread.allocate_lock()

IDLE's visual debugger has name-value panels for locals, including non-locals, and for globals.  It uses repr to gets value representations.  The locals panel is displayed by default.

Before line 174, 'lock' is bound to None, so before executing line 59, the display is "lock:None\nname:'unittest'".  After line 59, debugger apparently tries to get the repr of the in-process instance.  Since the call in 174 has not completed and should not have rebound 'lock' to the instance, I do not understand why.  (Brett, I now understand what you wrote to be pointing as this puzzle also.)  ppperry, additional light would be appreciated.

Given that debugger does try to get the repr of the instance, both Brett Cannon, here, and (ppperry), on duplicate issue #34609 (now closed), have pointed out that _ModuleLock.__repr__ uses self.name:
        return '_ModuleLock({!r}) at {}'.format(self.name, id(self))

I verified that updating the locals panel is the problem by starting over and turning the panel off until past the the assignment to self.name, at which point, the lock value is "_ModuleLock('unittest') at ...".

Debugger should be prepared for repr to fail, and display something informative.  In the present case, perhaps

repr raised "AttributeError: '_ModuleLock' object has no attribute 'name'"

with a check for whether the exception message contains "'.*' object" (and add such if not present).
msg326757 - (view) Author: (ppperry) Date: 2018-10-01 02:09
Line 59 isn't actually executed; the error comes from the trace event that gets fired before line 59, which is the first `line` event in the frame containing the uninitialized _ModuleLock.
History
Date User Action Args
2018-10-01 02:09:04ppperrysetnosy: + ppperry
messages: + msg326757
2018-09-23 20:32:27terry.reedysetnosy: - ppperry
title: IDLE debugger crashes when `repr` raises an exception -> IDLE debugger: failure stepping through module loading
messages: + msg326179

versions: + Python 3.7, Python 3.8
2018-09-23 18:29:35ppperrysettitle: IDLE debugger: problem importing user created module -> IDLE debugger crashes when `repr` raises an exception
2018-09-23 18:23:32ppperrysetnosy: + ppperry
2018-09-23 18:14:03terry.reedylinkissue34609 superseder
2018-04-12 16:36:04brett.cannonsetmessages: + msg315228
2018-04-11 23:56:10terry.reedysetnosy: + brett.cannon
type: crash -> behavior
messages: + msg315208
2018-04-11 22:50:54om364@setfiles: + PositionalList.py

nosy: + om364@
messages: + msg315203

type: behavior -> crash
2018-03-13 12:26:02terry.reedysettype: compile error -> behavior
title: debugger issue concerning importing user created modules into another program -> IDLE debugger: problem importing user created module
messages: + msg313744
stage: test needed
2018-03-13 05:02:16jcdlrcreate