test_gdb failing on 32-bit armv7l when built with GCC -Og: <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc #84923

gpshead · 2020-05-23T18:30:25Z

BPO	40746
Nosy	@gpshead, @vstinner, @ambv

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-01-28.19:25:54.119>
created_at = <Date 2020-05-23.18:30:25.238>
labels = ['tests', '3.9', '3.10']
title = "test_gdb failing on 32-bit armv7l when built with GCC -Og: <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc"
updated_at = <Date 2022-01-28.19:25:54.118>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2022-01-28.19:25:54.118>
actor = 'gregory.p.smith'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-28.19:25:54.119>
closer = 'gregory.p.smith'
components = ['Tests']
creation = <Date 2020-05-23.18:30:25.238>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 40746
keywords = []
message_count = 12.0
messages = ['369736', '371269', '371272', '371274', '372633', '373593', '376719', '376887', '401818', '401850', '401851', '412027']
nosy_count = 3.0
nosy_names = ['gregory.p.smith', 'vstinner', 'lukasz.langa']
pr_nums = []
priority = 'high'
resolution = 'works for me'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40746'
versions = ['Python 3.9', 'Python 3.10']

gpshead · 2020-05-23T18:30:25Z

https://buildbot.python.org/all/#/builders/727

test_tuples (test.test_gdb.PrettyPrintTests)
Verify the pretty-printing of tuples ... ok
test_bt (test.test_gdb.PyBtTests)
Verify that the "py-bt" command works ... FAIL
Stderr:
Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc:
Error occurred in Python command: Cannot access memory at address 0xfffffedc

and a pile of similar errors after test_bt

Marking release blocker as this isn't present in 3.8 or earlier, we've got a regression here.

i haven't tried to debug yet, but that pointer value smells like something did a negative offset from NULL...

vstinner · 2020-06-11T13:07:36Z

Some data from a recent build:
https://buildbot.python.org/all/#/builders/645/builds/480

That's 32-bit ARM, armv7l ABI with GCC 8.3 and GDB 8.2.

Python is built with -Og optimization level. In my experience, gdb fails to read debug symbols when Python is optimized at -Og level. Last time I asked a gdb developer, he told me that he always use -O0, so do I.

test.pythoninfo:

platform.architecture: 32bit ELF
platform.libc_ver: glibc 2.28
platform.platform: Linux-4.19.97-v7l+-armv7l-with-glibc2.28
platform.python_implementation: CPython

CC.version: gcc (Raspbian 8.3.0-6+rpi1) 8.3.0
sysconfig[PY_CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall
sysconfig[PY_CFLAGS_NODIST]: -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal
sysconfig[PY_STDMODULE_CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I. -I./Include

gdb_version: GNU gdb (Raspbian 8.2.1-2) 8.2.1

vstinner · 2020-06-11T13:20:17Z

Marking release blocker as this isn't present in 3.8 or earlier, we've got a regression here.

FYI we have issue with test_gdb on Fedora for at least 5 years. Most issues come from -Og optimization level of test_gdb.

Recently, I modified test_gdb to skip tests when we detect that gdb fails to read debug symbols, which happens often when using -Og: bpo-40019.

commit 7bf069b
Author: Victor Stinner <vstinner@python.org>
Date: Fri Mar 20 08:23:26 2020 +0100

bpo-40019: Skip test_gdb if Python was optimized (GH-19081)

test_gdb now skips tests if it detects that gdb failed to read debug
information because the Python binary is optimized.

vstinner · 2020-06-11T13:21:32Z

I closed bpo-17737 "test_gdb fails on armv7hl" as out of date.

ambv · 2020-06-29T21:02:12Z

This will miss 3.9.0b4.

ambv · 2020-07-13T11:34:51Z

Note: 3.9.0b5, the last beta before 3.9.0, is next week.

vstinner · 2020-09-11T10:55:39Z

gdb doesn't work fully when Python is built with -Og. I don't think that there is much that we can do. I suggest to skip test_gdb on 32-bit ARM when Python is built with -Og.

I don't think that this regular requires the "release blocker" priority. It would be nice to fix but it should not hold the 3.9.0 final release.

ambv · 2020-09-14T17:51:44Z

Downgrading since this missed rc1 anyway.

vstinner · 2021-09-15T09:18:59Z

test_gdb now pass on ARM Raspbian 3.x. I'm not sure why test_gdb fails on 3.10 but pass on 3.x.

For me, it looks more like a gcc or gdb issue, rather than a Python bug.

ARM Raspbian 3.10:
https://buildbot.python.org/all/#/builders/685/builds/76

test.pythoninfo:

CC.version: gcc (Raspbian 10.2.1-6+rpi1) 10.2.1 20210110
gdb_version: GNU gdb (Raspbian 10.1-1.7) 10.1.90.20210103-git

sysconfig[CCSHARED]: -fPIC
sysconfig[CC]: gcc -pthread
sysconfig[CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall
sysconfig[CONFIG_ARGS]: '--prefix' '/var/lib/buildbot/workers/3.10.gps-raspbian/build/target' '--with-pydebug'
sysconfig[HOST_GNU_TYPE]: armv7l-unknown-linux-gnueabihf
sysconfig[MACHDEP]: linux
sysconfig[MULTIARCH]: arm-linux-gnueabihf
sysconfig[OPT]: -g -Og -Wall
sysconfig[PY_CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall
sysconfig[PY_CFLAGS_NODIST]: -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal
sysconfig[PY_STDMODULE_CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I. -I./Include
sysconfig[Py_DEBUG]: 1

Logs:

======================================================================
FAIL: test_bt (test.test_gdb.PyBtTests)
Verify that the "py-bt" command works
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.10.gps-raspbian/build/Lib/test/test_gdb.py", line 776, in test_bt
    self.assertMultilineMatches(bt,
  File "/var/lib/buildbot/workers/3.10.gps-raspbian/build/Lib/test/test_gdb.py", line 295, in assertMultilineMatches
    self.fail(msg='%r did not match %r' % (actual, pattern))
AssertionError: 'Breakpoint 1 at 0x258ba4: file Python/bltinmodule.c, line 1194.\n[Thread debugging using libthread_db enabled]\nUsing host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".\n\nBreakpoint 1, builtin_id (self=, v=42) at Python/bltinmodule.c:1194\n1194\t{\nTraceback (most recent call first):\n  <built-in method id of module object at remote 0xb68b8360>\n' did not match '^.*\nTraceback \\(most recent call first\\):\n  <built-in method id of module object .*>\n  File ".*gdb_sample.py", line 10, in baz\n    id\\(42\\)\n  File ".*gdb_sample.py", line 7, in bar\n    baz\\(a, b, c\\)\n  File ".*gdb_sample.py", line 4, in foo\n    bar\\(a, b, c\\)\n  File ".*gdb_sample.py", line 12, in <module>\n    foo\\(1, 2, 3\\)\n'

Stderr:
Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0xfdfdfd83:
Error occurred in Python: Cannot access memory at address 0xfdfdfd83

gpshead · 2021-09-15T16:23:15Z

fwiw I updated my arm raspbian buildbot from raspbian Buster to raspbian Bullseye in the last few days. That could also explain the difference. More recent toolchain versions.

gpshead · 2021-09-15T16:24:42Z

although it looks like the 3.10 failure you linked to ran after that. so... the upgrade may not explain things.

gpshead · 2022-01-28T19:25:54Z

I manually tested this on 3.11 main and it appears to be working.

zhuofeng6 · 2023-02-24T03:49:22Z

Is this question OK now? Which commit was it repaired?

gpshead added release-blocker 3.9 only security fixes 3.10 only security fixes tests Tests in the Lib/test dir labels May 23, 2020

vstinner changed the title ~~test_gdb failing on Raspbian on 3.9, regression from 3.8~~ test_gdb failing on 32-bit armv7l when built with GCC -Og (fail on Raspbian on 3.9, regression from 3.8) Jun 11, 2020

ambv removed the release-blocker label Sep 14, 2020

vstinner changed the title ~~test_gdb failing on 32-bit armv7l when built with GCC -Og (fail on Raspbian on 3.9, regression from 3.8)~~ test_gdb failing on 32-bit armv7l when built with GCC -Og: <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc Sep 15, 2021

gpshead closed this as completed Jan 28, 2022

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_gdb failing on 32-bit armv7l when built with GCC -Og: <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc #84923

test_gdb failing on 32-bit armv7l when built with GCC -Og: <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc #84923

gpshead commented May 23, 2020

gpshead commented May 23, 2020

vstinner commented Jun 11, 2020

vstinner commented Jun 11, 2020

vstinner commented Jun 11, 2020

ambv commented Jun 29, 2020

ambv commented Jul 13, 2020

vstinner commented Sep 11, 2020

ambv commented Sep 14, 2020

vstinner commented Sep 15, 2021

gpshead commented Sep 15, 2021

gpshead commented Sep 15, 2021

gpshead commented Jan 28, 2022

zhuofeng6 commented Feb 24, 2023

test_gdb failing on 32-bit armv7l when built with GCC -Og: <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc #84923

test_gdb failing on 32-bit armv7l when built with GCC -Og: <class 'gdb.MemoryError'> Cannot access memory at address 0xfffffedc #84923

Comments

gpshead commented May 23, 2020

gpshead commented May 23, 2020

vstinner commented Jun 11, 2020

vstinner commented Jun 11, 2020

vstinner commented Jun 11, 2020

ambv commented Jun 29, 2020

ambv commented Jul 13, 2020

vstinner commented Sep 11, 2020

ambv commented Sep 14, 2020

vstinner commented Sep 15, 2021

gpshead commented Sep 15, 2021

gpshead commented Sep 15, 2021

gpshead commented Jan 28, 2022

zhuofeng6 commented Feb 24, 2023