classification
Title: AMD64 RHEL8 LTO + PGO 3.x build failed with: /usr/bin/ld: Dwarf Error: Offset (2487097600) greater than or equal to .debug_str size (571933).
Type: Stage: resolved
Components: Build Versions: Python 3.11
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: corona10, cstratak, pablogsal, vstinner
Priority: normal Keywords:

Created on 2021-06-09 07:50 by vstinner, last changed 2021-09-21 18:39 by vstinner. This issue is now closed.

Messages (4)
msg395386 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-09 07:50
On RHEL8, gcc (GCC) 8.4.1 20200928 (Red Hat 8.4.1-1) fails to build Python with LTO+PGO: the linker fails with "Dwarf Error".

On ~200 builds, the linker error only occurred once (build 279) :-( It sounds really hard to reproduce.

I suggest to close the issue as "out of date" if it doesn't come back next weeks.

https://buildbot.python.org/all/#/builders/568/builds/279

gcc -pthread   -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g -fprofile-generate -Xlinker -export-dynamic -o Programs/_testembed Programs/_testembed.o libpython3.11.a -lcrypt -lpthread -ldl  -lutil -lm   -lm 
/usr/bin/ld: Dwarf Error: Offset (2487097600) greater than or equal to .debug_str size (571933).
/usr/bin/ld: Dwarf Error: Could not find abbrev number 12095.

Then the linker failed to find many symbols:

/tmp/cccV00H7.lto.o: In function `signal_pthread_sigmask':
<artificial>:(.text+0x292cbc): undefined reference to `_Py_Sigset_Converter'
/tmp/cccV00H7.lto.o: In function `fill_siginfo':
<artificial>:(.text+0x2930cb): undefined reference to `_PyLong_FromUid'
/tmp/cccV00H7.lto.o: In function `signal_sigwaitinfo':
<artificial>:(.text+0x29325e): undefined reference to `_Py_Sigset_Converter'
/tmp/cccV00H7.lto.o: In function `signal_sigtimedwait':
<artificial>:(.text+0x293566): undefined reference to `_Py_Sigset_Converter'
/tmp/cccV00H7.lto.o: In function `signal_sigwait':
<artificial>:(.text+0x293af0): undefined reference to `_Py_Sigset_Converter'
/tmp/cccV00H7.lto.o: In function `PyUnicode_FSDecoder':
<artificial>:(.text+0x315fed): undefined reference to `PyOS_FSPath'
/tmp/cccV00H7.lto.o: In function `mkpwent':
<artificial>:(.text+0x31a78c): undefined reference to `_PyLong_FromUid'
<artificial>:(.text+0x31a7a1): undefined reference to `_PyLong_FromGid'
/tmp/cccV00H7.lto.o: In function `pwd_getpwuid':
<artificial>:(.text+0x31afb6): undefined reference to `_Py_Uid_Converter'
<artificial>:(.text+0x31b1df): undefined reference to `_PyLong_FromUid'
/tmp/cccV00H7.lto.o: In function `PyUnicode_FSConverter':
<artificial>:(.text+0x349247): undefined reference to `PyOS_FSPath'
/tmp/cccV00H7.lto.o: In function `_io_open':
<artificial>:(.text+0x38781d): undefined reference to `PyOS_FSPath'
/tmp/cccV00H7.lto.o:(.data+0x74528): undefined reference to `PyInit_posix'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:603: python] Error 1
make[3]: *** Waiting for unfinished jobs....
/usr/bin/ld: Dwarf Error: Could not find abbrev number 6864.
/tmp/ccMUbctW.lto.o: In function `fill_siginfo':
<artificial>:(.text+0x1f010b): undefined reference to `_PyLong_FromUid'
/tmp/ccMUbctW.lto.o: In function `signal_sigwait':
<artificial>:(.text+0x2a6d80): undefined reference to `_Py_Sigset_Converter'
/tmp/ccMUbctW.lto.o: In function `signal_sigwaitinfo':
<artificial>:(.text+0x2a821e): undefined reference to `_Py_Sigset_Converter'
/tmp/ccMUbctW.lto.o: In function `signal_pthread_sigmask':
<artificial>:(.text+0x2b05bc): undefined reference to `_Py_Sigset_Converter'
/tmp/ccMUbctW.lto.o: In function `signal_sigtimedwait':
<artificial>:(.text+0x2b2536): undefined reference to `_Py_Sigset_Converter'
/tmp/ccMUbctW.lto.o: In function `PyUnicode_FSDecoder':
<artificial>:(.text+0x30d08d): undefined reference to `PyOS_FSPath'
/tmp/ccMUbctW.lto.o: In function `mkpwent':
<artificial>:(.text+0x30edac): undefined reference to `_PyLong_FromUid'
<artificial>:(.text+0x30edc1): undefined reference to `_PyLong_FromGid'
/tmp/ccMUbctW.lto.o: In function `pwd_getpwuid':
<artificial>:(.text+0x30f5d6): undefined reference to `_Py_Uid_Converter'
<artificial>:(.text+0x30f7ff): undefined reference to `_PyLong_FromUid'
/tmp/ccMUbctW.lto.o: In function `PyUnicode_FSConverter':
<artificial>:(.text+0x33b157): undefined reference to `PyOS_FSPath'
/tmp/ccMUbctW.lto.o: In function `_io_open':
<artificial>:(.text+0x377d0d): undefined reference to `PyOS_FSPath'
/tmp/ccMUbctW.lto.o:(.data+0x84748): undefined reference to `PyInit_posix'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:744: Programs/_testembed] Error 1
make[3]: Leaving directory '/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build'
make[2]: *** [Makefile:533: build_all_generate_profile] Error 2
make[2]: Leaving directory '/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build'
make[1]: *** [Makefile:509: profile-gen-stamp] Error 2
make[1]: Leaving directory '/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build'
make: *** [Makefile:521: profile-run-stamp] Error 2
program finished with exit code 2
elapsedTime=157.205212
msg395395 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-09 08:31
On OFTC #gcc, I'm told that it can this known issue fixed in 2018:
https://sourceware.org/bugzilla/show_bug.cgi?id=23425
msg395400 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-09 09:03
I reported the compiler bug to RHEL: https://bugzilla.redhat.com/show_bug.cgi?id=1969775
msg402346 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-21 18:39
> I reported the compiler bug to RHEL: https://bugzilla.redhat.com/show_bug.cgi?id=1969775

I close the issue. It's now tracked at https://bugzilla.redhat.com/show_bug.cgi?id=1969775
History
Date User Action Args
2021-09-21 18:39:23vstinnersetstatus: open -> closed
resolution: third party
messages: + msg402346

stage: resolved
2021-06-21 15:37:08vstinnersetnosy: + cstratak
2021-06-09 16:57:38corona10setnosy: + corona10
2021-06-09 09:03:41vstinnersetmessages: + msg395400
2021-06-09 08:31:50vstinnersetmessages: + msg395395
2021-06-09 07:50:20vstinnercreate