classification
Title: Test hang causes --enable-optimizations build to hang
Type: Stage: resolved
Components: Build Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Jarek Zgoda, gregory.p.smith, ned.deily
Priority: normal Keywords:

Created on 2019-07-10 11:17 by Jarek Zgoda, last changed 2019-08-08 18:51 by Jarek Zgoda. This issue is now closed.

Files
File name Uploaded Description Edit
pgo_task_log.txt Jarek Zgoda, 2019-07-10 11:17
test.pythoninfo.txt Jarek Zgoda, 2019-07-11 07:48
3.7.4-test_output.txt Jarek Zgoda, 2019-07-11 07:50
test.pythoninfo.libressl-2.9.2.txt Jarek Zgoda, 2019-07-30 13:10 test.pythoninfo output against libressl-2.9.2
Messages (10)
msg347616 - (view) Author: Jarek Zgoda (Jarek Zgoda) * Date: 2019-07-10 11:17
I'm trying to build Python-3.7.4 from release tarball.

Configure line: ./configure --prefix=/opt/python-3.7.4 --disable-shared  --enable-optimizations

System: Ubuntu 18.04.2 amd64, 4.15.0-54-generic #58-Ubuntu SMP

Build fails.
msg347623 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-07-10 15:02
Sorry you are having problems.  Suggest first trying a clean build without --enable-optimizations.  If that works, please supply the output from:

./python -m test.pythoninfo
msg347624 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-07-10 15:05
And, if the unoptimized build succeeds, also try running the tests with it:

make test
msg347658 - (view) Author: Jarek Zgoda (Jarek Zgoda) * Date: 2019-07-11 07:48
test.pythoninfo output
msg347659 - (view) Author: Jarek Zgoda (Jarek Zgoda) * Date: 2019-07-11 07:50
make test output from unoptimized build with --disable-shared --prefix=/opt/python-3.7.4
msg348246 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-07-21 14:40
Thanks for the additional info.  I see two issues here.  One, you are building with OpenSSL 1.1.1, the initial release back in 2018-09.  There have been a few releases since then (currently at 1.1.1c) which do fix problems.  You should try to upgrade it.

Second, and more importantly, is that "make run_profile_task" hangs because a test hangs up and the tests are being run sequentially (i.e. without regrtest -j options meaning tests all run in one process sequentially).  These days nearly all our testing using -j because of the possibility of individual tests failures and residual effects between tests. And note that "make test", which runs in parallel, did complete. I see there is some discussion going on Issue36044 about changing the set of tests run_profile_task uses which would likely mitigate problems like this.  I'm CCing gps here for his views.

It also appears Issue33467 is a similar report.  I'm going to close it as a duplicate of this one.
msg348259 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-07-21 18:17
While test hangs shouldn't happen, I recommend working around such odd situations you are finding on your system by supplying your own modified PROFILE_TASK setting at make time that avoids whatever test(s) are causing you problems.  The linked to issue36044 contains some suggested PROFILE_TASK settings.
msg348323 - (view) Author: Jarek Zgoda (Jarek Zgoda) * Date: 2019-07-23 08:52
Thank you for the information.

The build system is standard Ubuntu 18.04.2 and openssl is system provided. I will try to build Python against other openssl versions, as well against libressl.
msg348819 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-07-31 17:59
Jerek, is there anything more you are expecting us to do for this issue?  The only thing I can think of at the moment might be to add something to the "Profile Guided Optimization" section of the top-level README.rst to include @gps's suggestions for tailoring PROFILE_TASK.
msg349248 - (view) Author: Jarek Zgoda (Jarek Zgoda) * Date: 2019-08-08 18:51
Not really. I tried with limited test list and I keep getting the exact same failure in test_logging, moreover build passes all tests on all other machines we have. I guess this is some weird isolated issue not related to Python at all so I'm closing it as "not a bug". Thank you for your help.
History
Date User Action Args
2019-08-08 18:51:04Jarek Zgodasetstatus: open -> closed
resolution: not a bug
messages: + msg349248

stage: resolved
2019-07-31 17:59:50ned.deilysetmessages: + msg348819
2019-07-30 13:10:44Jarek Zgodasetfiles: + test.pythoninfo.libressl-2.9.2.txt
2019-07-23 08:52:33Jarek Zgodasetmessages: + msg348323
2019-07-21 18:17:11gregory.p.smithsetmessages: + msg348259
title: Test failures cause --enable-optimizations build to hang -> Test hang causes --enable-optimizations build to hang
2019-07-21 14:47:04ned.deilysettitle: Multiple test failures during build -> Test failures cause --enable-optimizations build to hang
2019-07-21 14:44:18ned.deilylinkissue33467 superseder
2019-07-21 14:40:34ned.deilysetnosy: + gregory.p.smith
messages: + msg348246
2019-07-11 07:50:19Jarek Zgodasetfiles: + 3.7.4-test_output.txt

messages: + msg347659
2019-07-11 07:48:53Jarek Zgodasetfiles: + test.pythoninfo.txt

messages: + msg347658
2019-07-10 15:05:42ned.deilysetmessages: + msg347624
2019-07-10 15:02:55ned.deilysetnosy: + ned.deily
messages: + msg347623
2019-07-10 11:17:58Jarek Zgodacreate