classification
Title: test_gdb failing on 32-bit armv7l when built with GCC -Og (fail on Raspbian on 3.9, regression from 3.8)
Type: Stage:
Components: Tests Versions: Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gregory.p.smith, lukasz.langa, vstinner
Priority: high Keywords:

Created on 2020-05-23 18:30 by gregory.p.smith, last changed 2020-09-14 17:51 by lukasz.langa.

Messages (8)
msg369736 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2020-05-23 18:30
https://buildbot.python.org/all/#/builders/727

test_tuples (test.test_gdb.PrettyPrintTests)
Verify the pretty-printing of tuples ... ok
test_bt (test.test_gdb.PyBtTests)
Verify that the "py-bt" command works ... FAIL
Stderr:
Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc: 
Error occurred in Python command: Cannot access memory at address 0xfffffedc

and a pile of similar errors after test_bt

Marking release blocker as this isn't present in 3.8 or earlier, we've got a regression here.

i haven't tried to debug yet, but that pointer value smells like something did a negative offset from NULL...
msg371269 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-11 13:07
Some data from a recent build:
https://buildbot.python.org/all/#/builders/645/builds/480

That's 32-bit ARM, armv7l ABI with GCC 8.3 and GDB 8.2.

Python is built with -Og optimization level. In my experience, gdb fails to read debug symbols when Python is optimized at -Og level. Last time I asked a gdb developer, he told me that he always use -O0, so do I.


test.pythoninfo:

platform.architecture: 32bit ELF
platform.libc_ver: glibc 2.28
platform.platform: Linux-4.19.97-v7l+-armv7l-with-glibc2.28
platform.python_implementation: CPython

CC.version: gcc (Raspbian 8.3.0-6+rpi1) 8.3.0
sysconfig[PY_CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall
sysconfig[PY_CFLAGS_NODIST]: -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal
sysconfig[PY_STDMODULE_CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I. -I./Include

gdb_version: GNU gdb (Raspbian 8.2.1-2) 8.2.1
msg371272 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-11 13:20
> Marking release blocker as this isn't present in 3.8 or earlier, we've got a regression here.

FYI we have issue with test_gdb on Fedora for at least 5 years. Most issues come from -Og optimization level of test_gdb.

Recently, I modified test_gdb to skip tests when we detect that gdb fails to read debug symbols, which happens often when using -Og: bpo-40019.

commit 7bf069b6110278102c8f4719975a5eb5a5af25f9
Author: Victor Stinner <vstinner@python.org>
Date:   Fri Mar 20 08:23:26 2020 +0100

    bpo-40019: Skip test_gdb if Python was optimized (GH-19081)
    
    test_gdb now skips tests if it detects that gdb failed to read debug
    information because the Python binary is optimized.
msg371274 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-11 13:21
I closed bpo-17737 "test_gdb fails on armv7hl" as out of date.
msg372633 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-06-29 21:02
This will miss 3.9.0b4.
msg373593 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-07-13 11:34
Note: 3.9.0b5, the last beta before 3.9.0, is next week.
msg376719 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-11 10:55
gdb doesn't work fully when Python is built with -Og. I don't think that there is much that we can do. I suggest to skip test_gdb on 32-bit ARM when Python is built with -Og.

I don't think that this regular requires the "release blocker" priority. It would be nice to fix but it should not hold the 3.9.0 final release.
msg376887 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-09-14 17:51
Downgrading since this missed rc1 anyway.
History
Date User Action Args
2020-09-14 17:51:43lukasz.langasetpriority: release blocker -> high

messages: + msg376887
2020-09-11 10:55:39vstinnersetmessages: + msg376719
2020-07-13 11:34:51lukasz.langasetmessages: + msg373593
2020-06-29 21:02:11lukasz.langasetnosy: + lukasz.langa
messages: + msg372633
2020-06-11 13:21:32vstinnersetmessages: + msg371274
2020-06-11 13:20:17vstinnersetmessages: + msg371272
2020-06-11 13:18:54vstinnersettitle: test_gdb failing on Raspbian on 3.9, regression from 3.8 -> test_gdb failing on 32-bit armv7l when built with GCC -Og (fail on Raspbian on 3.9, regression from 3.8)
2020-06-11 13:07:35vstinnersetnosy: + vstinner
messages: + msg371269
2020-05-23 18:30:25gregory.p.smithcreate