This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_gdb failures
Type: behavior Stage: needs patch
Components: Tests Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, dmalcolm, larry, pitrou, python-dev
Priority: normal Keywords:

Created on 2013-11-24 01:10 by larry, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
gdb_test_failure larry, 2013-11-24 01:10
Messages (17)
msg204143 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-24 01:10
test_gdb started failing on me within the past few days.  (I would have reported it sooner but I figured the omniscient CPython core dev community already knew about it.)

The machine it's failing on is Ubuntu 13.10, 64-bit.
msg204144 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-24 01:11
Please see the attached output from the test run.  But here's a sample:

======================================================================
FAIL: test_lists (test.test_gdb.PrettyPrintTests)
Verify the pretty-printing of lists
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/larry/src/python/nodocstrings/Lib/test/test_gdb.py", line 258, in test_lists
    self.assertGdbRepr(list(range(5)))
  File "/home/larry/src/python/nodocstrings/Lib/test/test_gdb.py", line 228, in assertGdbRepr
    cmds_after_breakpoint)
  File "/home/larry/src/python/nodocstrings/Lib/test/test_gdb.py", line 196, in get_gdb_repr
    import_site=import_site)
  File "/home/larry/src/python/nodocstrings/Lib/test/test_gdb.py", line 179, in get_stack_trace
    self.assertEqual(unexpected_errlines, [])
AssertionError: Lists differ: ["Python Exception <class 'UnicodeDecodeEr[197 chars]e: "] != []

First list contains 2 additional elements.
First extra element 0:
Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0xfc in position 1: invalid start byte: 

+ []
- ["Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode "
-  'byte 0xfc in position 1: invalid start byte: ',
-  "Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode "
-  'byte 0xfc in position 1: invalid start byte: ']
msg204145 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-11-24 01:11
> test_gdb started failing on me within the past few days

Can you please bisect to find the offending changeset?
msg204150 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-24 01:47
The first bad revision is:
changeset:   87426:9f2a0043396b
user:        Antoine Pitrou <solipsis@pitrou.net>
date:        Sat Nov 23 17:40:36 2013 +0100
files:       Lib/test/test_gdb.py Tools/gdb/libpython.py
description:
Issue #19308: fix the gdb plugin on gdbs linked with Python 3
msg204153 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-11-24 01:52
If you go back one revision, test_gdb isn't run at all, right?
(it should be skipped)
msg204154 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-24 01:52
That's correct, it is skipped.
msg204156 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-11-24 01:57
Ok, it seems the failures are due to a non-pydebug build. I'll investigate tomorrow.
msg204157 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-24 01:59
The test passes if I turn on "--with-pydebug".  And Antoine told me in IRC that he sees the failure if he turns off "--with-pydebug".  So we're in business.
msg204218 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-11-24 13:58
New changeset 7a14cde3c4df by Antoine Pitrou in branch 'default':
Issue #19743: fix test_gdb on some optimized Python builds
http://hg.python.org/cpython/rev/7a14cde3c4df
msg204219 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-11-24 14:01
New changeset e42b449f73fd by Antoine Pitrou in branch '3.3':
Issue #19743: fix test_gdb on some optimized Python builds
http://hg.python.org/cpython/rev/e42b449f73fd
msg204220 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-24 14:03
Here's what I've figured out.  If I build trunk then run:

% gdb --batch -iex "add-auto-load-safe-path /home/larry/src/python/buildtrunk/python-gdb.py" --eval-command="set breakpoint pending yes" --eval-command="break builtin_id" --eval-command=run --eval-command=backtrace --args /home/larry/src/python/buildtrunk/python -S -c "id([0, 1, 2, 3, 4])"

I see UnicodeDecodeErrors spat out in the middle of the traceback:

#0  builtin_id (self=<module at remote 0x7ffff7fbfc28>, v=[0, 1, 2, 3, 4]) at Python/bltinmodule
.c:991
#1  0x00000000004d8695 in call_function (oparg=<optimized out>, pp_stack=0x7fffffffdc00) at Pyth
on/ceval.c:4212
#2  PyEval_EvalFrameEx (f=f@entry=Frame 0x7ffff7f3e788, for file <string>, line 1, in <module> (
), throwflag=throwflag@entry=0) at Python/ceval.c:2826
Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0x80 in position 0
: invalid start byte: 
Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0x80 in position $: invalid start byte: 
#3  0x00000000004d9809 in PyEval_EvalCodeEx (closure=0x0, [...]

If I run the same sequence of commands manually (remove the --batch and the batched commands from the gdb command-line, then type them in myself) I still see the decode error.

If I then set a breakpoint on PyUnicodeDecodeError_Create() and run backtrace again, it still prints the decode errors, but it doesn't stop on the breakpoint.  (I guess this would be like getting to the assault on the ski chalet in Inception?)

Most worrisome of all: if I change the Makefile from -O3 to -O0, the decode errors go away.
msg204221 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-24 14:05
Sorry, both exceptions in the output of the traceback are:

Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte:
msg204222 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-24 14:08
Antoine: your checked-in workaround makes the test pass.  But I'm leaving this open because I'm still hoping David will take a look into it.
msg204223 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-11-24 14:09
Agreed.
msg204353 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2013-11-25 15:53
FWIW, I feel that it's worth just expecting failures with an *optimized* build: with an optimizing compiler, there's likely to always be some program counter location where the debugger is going to get confused for some variables.  Given umpteen different compiler variants, debugger variants etc, this is never going to be perfect.

So IMHO we should run the test on a --pydebug build, but accept that there will be some failures when optimization is on.
msg204393 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-25 20:14
If skipping them for optimized builds is the right call, then aren't we done here?  Can we close this bug as fixed?
msg225167 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-08-10 21:56
Seems as if this can be closed.
History
Date User Action Args
2022-04-11 14:57:54adminsetgithub: 63942
2014-08-11 01:15:21pitrousetstatus: open -> closed
resolution: fixed
2014-08-10 21:56:26BreamoreBoysetnosy: + BreamoreBoy
messages: + msg225167
2013-11-25 20:14:20larrysetmessages: + msg204393
2013-11-25 15:53:37dmalcolmsetmessages: + msg204353
2013-11-24 14:09:17pitrousetmessages: + msg204223
2013-11-24 14:08:59larrysetmessages: + msg204222
2013-11-24 14:05:08larrysetmessages: + msg204221
2013-11-24 14:03:54larrysetmessages: + msg204220
2013-11-24 14:01:48python-devsetmessages: + msg204219
2013-11-24 13:59:56pitrousetnosy: + dmalcolm
2013-11-24 13:58:24python-devsetnosy: + python-dev
messages: + msg204218
2013-11-24 01:59:15larrysetmessages: + msg204157
2013-11-24 01:57:53pitrousetmessages: + msg204156
2013-11-24 01:52:59larrysetmessages: + msg204154
2013-11-24 01:52:36pitrousetmessages: + msg204153
2013-11-24 01:47:38larrysetmessages: + msg204150
2013-11-24 01:11:37pitrousetmessages: + msg204145
2013-11-24 01:11:24larrysetmessages: + msg204144
2013-11-24 01:10:48larrycreate