This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: IDLE debugger fails with non-trivial __new__ super call
Type: behavior Stage: test needed
Components: IDLE Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: Camion, ppperry, terry.reedy
Priority: normal Keywords:

Created on 2017-11-26 11:46 by Camion, last changed 2022-04-11 14:58 by admin.

Messages (3)
msg307001 - (view) Author: (Camion) Date: 2017-11-26 11:46
Hello all,

I have been investigating a bug with a generator, which crashed, reporting that "TypeError: 'Fraction' object is not iterable".

Then I have tried to find it with the debugger and/or to reproduce it with a simpler program, but, I have not managed to reproduce the bug in my test program and I have been stopped by the fact that the debugger crashes with another error:

I have reproduced it on version 3.4.2 ans 3.6.3, BUT NOT on version 2.7.9. So I suspect it has been introduced with version 3.

Here is how to reproduce it : 

1/ Open Idle
2/ create and save this text program (or open if you already did - or don't do anything if it's already opened X-D)

from fractions import Fraction
U=Fraction(1)

def test(x):
    for i in range(1, x.denominator):
        x*=x
        yield i, i*x

for m, n in test(U*3/5):
    print (m, n)

3/ Run/check module (F5) in Idle editors window : it works perfectly
4/ start the debugger from Idle menus
5/ activate the "source" checkbox
6/ Run/check module (F5) in Idle editors window
7/ Click 3 times on [over] to get to the "for m, n in test(U*3/5):" line
8/ Click 6 times on [step], to get to the line 116 ("for m, n in test(U*3/5):") in the fractions.py files
9/ Click a seventh time on [step] and your program will crash with the folowing error (here with 3.6.3)

Traceback (most recent call last):
  File "/raid/ArcFolder/Mes documents/Mes Textes/Mes programmes/Machin/Test_itérateur_et_fractions.py", line 12, in <module>
    for m, n in test(U*3/5):
  File "/usr/local/lib/python3.6/fractions.py", line 376, in forward
    return monomorphic_operator(a, b)
  File "/usr/local/lib/python3.6/fractions.py", line 419, in _mul
    return Fraction(a.numerator * b.numerator, a.denominator * b.denominator)
  File "/usr/local/lib/python3.6/fractions.py", line 117, in __new__
    if denominator is None:
  File "/usr/local/lib/python3.6/fractions.py", line 117, in __new__
    if denominator is None:
  File "/usr/local/lib/python3.6/bdb.py", line 48, in trace_dispatch
    return self.dispatch_line(frame)
  File "/usr/local/lib/python3.6/bdb.py", line 66, in dispatch_line
    self.user_line(frame)
  File "/usr/local/lib/python3.6/idlelib/debugger.py", line 24, in user_line
    self.gui.interaction(message, frame)
AttributeError: _numerator5/ Run/check module (F5) in Idle editors window

I have observed once 3.4.2 that the same operation without activating the source checkbox, lead to a full idle freeze, but I have not been able to reproduce it.
msg307027 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-11-26 22:36
This is not a crash because IDLE does not crash.  It does not even exit with a traceback.  Some crash and freeze bugs have been fixed since 3.4.2.

I verified with 3.7.0a2 on Win10.  The failing example can be simplified to the first two lines.  Step over the import and into the Fraction call to get to fractions.Fraction.__new__, line 115
        self = super(Fraction, cls).__new__(cls)
(Step 8 in the report above has the wrong code.)  Step 2 gives this simplified traceback:

Traceback (most recent call last):
  File "F:\Python\a\tem2.py", line 3, in <module>
    U=Fraction(1)
  File "C:\Programs\Python37\lib\fractions.py", line 117, in __new__
    if denominator is None:
  File "C:\Programs\Python37\lib\fractions.py", line 117, in __new__
    if denominator is None:
  File "C:\Programs\Python37\lib\bdb.py", line 86, in trace_dispatch
    return self.dispatch_line(frame)
  File "C:\Programs\Python37\lib\bdb.py", line 110, in dispatch_line
    self.user_line(frame)
  File "C:\Programs\Python37\lib\idlelib\debugger.py", line 24, in user_line
    self.gui.interaction(message, frame)
AttributeError: _numerator

If I replace the line above with
        new = super(Fraction, cls)  # resolves to object.__new__
        self = new(cls)
stepping the first works.  For the new call, hitting 'step' or 'over' gives essentially the same traceback as above.  Hitting 'go' runs the code to completion.

If I add 'print('xxxx') after the new call, the print call shows up in the traceback, twice, but is not executed.

If I simplify the code to

class C():
    def __new__(cls):
        self = object.__new__(cls)
        return self
c = C()

and step into the C() code the the __new__ code, there is no problem.  Hence the revised title.

If I add slots make C, as with Fraction, there is still no problem.

class C():
	__slots__ = ('a',)
	def __new__(cls, a):
		self = object.__new__(cls)
		self.a = a
		return self
c = C(1)

Puzzles:
1. Why does the next line, 'if denominator...' show up in the traceback?  It does not have a function call.  Why is it printed twice?
2. Where does the _numerator error happen?  It is only set, never accessed.  Setting should work because _numerator is one of the two slots.
3. What extra feature of Fraction results in the failure.

If I leave IDLE's debugger off and invoke pdb by adding
import pdb; pbd.set_trace()
as line 2, before the Fraction call, stepping with s works on the line where Debugger failed.  Until a fix is released, use this (and/or print) as a workaround for code that makes debugger croak.

I do not now know how to write a fully automated unittest for debugger.  But it does not now even have a human-driven htest, and it should.
msg307540 - (view) Author: (ppperry) Date: 2017-12-04 03:43
Simplified reproducer for same bug without any imports:

class BadRepr:
	def __repr__(self):
		1/0
def broken():
    x=BadRepr()
    x=x #filler line for debugger

In this case, the problematic BadRepr object in the "broken" function explictly has a broken __repr__, but the uninitialized Fraction object in the "__new__" function of the Fraction class has a similarly broken __repr__
History
Date User Action Args
2022-04-11 14:58:54adminsetgithub: 76321
2017-12-04 03:43:40ppperrysetnosy: + ppperry
messages: + msg307540
2017-11-26 22:36:44terry.reedysetversions: + Python 3.7
title: Probable bug in all python3 / idle3 debugger -> IDLE debugger fails with non-trivial __new__ super call
messages: + msg307027

type: crash -> behavior
stage: test needed
2017-11-26 11:46:13Camioncreate