classification
Title: Buildbot s390x Fedora LTO + PGO 3.x fails intermittently
Type: Stage: resolved
Components: Demos and Tools Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: Nosy List: kj, pablogsal, vstinner, zach.ware
Priority: normal Keywords:

Created on 2020-12-24 19:36 by kj, last changed 2020-12-25 02:23 by kj. This issue is now closed.

Messages (3)
msg383702 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2020-12-24 19:36
Dear core developers, I noticed that for many recent commits, the s390x Fedora LTO + PGO 3.x buildbot often fails. Here's an error log::


gcc -pthread   -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g  -Xlinker -export-dynamic -o python Programs/python.o libpython3.10.a -lcrypt -lpthread -ldl  -lutil -lm   -lm 
gcc -pthread   -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g  -Xlinker -export-dynamic -o Programs/_testembed Programs/_testembed.o libpython3.10.a -lcrypt -lpthread -ldl  -lutil -lm   -lm 
/usr/bin/ld: python.lto.o: in function `run_mod':
/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Python/pythonrun.c:1230: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: python.lto.o: in function `builtin_compile':
/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Python/bltinmodule.c:808: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: python.lto.o: in function `Py_CompileStringObject':
/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Python/pythonrun.c:1307: undefined reference to `PyAST_CompileObject'
/usr/bin/ld: python.lto.o: in function `symtable_lookup':
/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1004: undefined reference to `_Py_Mangle'
/usr/bin/ld: python.lto.o: in function `symtable_record_directive':
/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1161: undefined reference to `_Py_Mangle'
/usr/bin/ld: python.lto.o: in function `type_new':
/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Objects/typeobject.c:2544: undefined reference to `_Py_Mangle'
/usr/bin/ld: python.lto.o: in function `symtable_add_def_helper':
/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Python/symtable.c:1018: undefined reference to `_Py_Mangle'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:592: python] Error 1
make[1]: *** Waiting for unfinished jobs....

[.............. I truncated the rest]

I have a hunch the error is caused by a lack of the '-fprofile-generate' flag for the first two gcc commands. The only issue is that it's sometimes there, and sometimes not, and I'm not familiar enough with the buildbot code to find out why. Sorry.

Eg. this commit https://github.com/python/cpython/commit/cc3467a57b61b0e7ef254b36790a1c44b13f2228 has s390x Fedora LTO + PGO 3.x succeeding, the 2 gcc lines also have the '-fprofile-generate' flag.

This commit https://github.com/python/cpython/commit/c6c43b28746b0642cc3c49dd8138b896bed3028f has s390x Fedora LTO + PGO 3.x failing, the 2 gcc lines have a blank space in place of the '-fprofile-generate' flag.

I could also be completely wrong and off track, so please feel free to correct me :), thanks.
msg383706 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-12-24 20:21
Isn't this the same as the problem described in https://bugs.python.org/issue42164 ?
msg383709 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2020-12-25 02:23
Oops, you're right! Sorry, I'll close this issue. Anyways I realized I had the wrong hunch - I was looking at a different part of the logs.
History
Date User Action Args
2020-12-25 02:23:14kjsetstatus: open -> closed
resolution: duplicate
messages: + msg383709

stage: resolved
2020-12-24 20:21:46pablogsalsetmessages: + msg383706
2020-12-24 19:36:20kjcreate