Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aarch64 RHEL7 LTO + PGO 3.7: "make" hangs when running test_asyncio #87197

Closed
vstinner opened this issue Jan 26, 2021 · 5 comments
Closed

aarch64 RHEL7 LTO + PGO 3.7: "make" hangs when running test_asyncio #87197

vstinner opened this issue Jan 26, 2021 · 5 comments
Labels
3.7 (EOL) end of life tests Tests in the Lib/test dir

Comments

@vstinner
Copy link
Member

BPO 43031
Nosy @vstinner
PRs
  • bpo-43031: Set a timeout when running tests in PGO build #24339
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2021-01-27.10:20:17.723>
created_at = <Date 2021-01-26.13:46:19.856>
labels = ['3.7', 'tests']
title = 'aarch64 RHEL7 LTO + PGO 3.7: "make" hangs when running test_asyncio'
updated_at = <Date 2021-01-27.10:20:17.723>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-01-27.10:20:17.723>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-01-27.10:20:17.723>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2021-01-26.13:46:19.856>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 43031
keywords = ['patch']
message_count = 5.0
messages = ['385712', '385716', '385724', '385752', '385753']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = ['24339']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue43031'
versions = ['Python 3.7']
    @vstinner
    Copy link
    Member Author

    test_asyncio hangs randomly on Python 3.7 on aarch64 RHEL7 LTO + PGO 3.7. The symptom is a failed build failing with:

    retry lost connection compile (retry)

    Full error:

    remoteFailed: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.

    Example of failed build:

    https://buildbot.python.org/all/#/builders/42/builds/4703

    Configure command:

    ./configure --prefix '$(PWD)/target' --with-lto --enable-optimizations

    Compile command:

    make -j10 all

    End of the make output:
    --------
    make run_profile_task
    make[1]: Entering directory `/home/buildbot/buildarea/3.7.cstratak-RHEL7-aarch64.lto-pgo/build'
    ./python -m test.regrtest --pgo || true
    0:00:00 load avg: 1.07 Run tests sequentially
    0:00:00 load avg: 1.07 [ 1/416] test_grammar
    (...)
    0:01:40 load avg: 1.07 [ 24/416] test_asynchat
    0:01:42 load avg: 1.07 [ 25/416] test_asyncio

    remoteFailed: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.
    ]
    --------

    I ran manually "./configure --prefix '$(PWD)/target' --with-lto --enable-optimizations && make -j10 all" twice on the worker directly, but test_asyncio passed successfully (I tried it twice).

    @vstinner vstinner added 3.7 (EOL) end of life tests Tests in the Lib/test dir labels Jan 26, 2021
    @vstinner
    Copy link
    Member Author

    Python 3.7 no longer accept bugfixes, only security fixes:
    https://devguide.python.org/#status-of-python-branches

    Maybe we should just remove this PGO worker (but keep other Python 3.7 workers).

    @vstinner
    Copy link
    Member Author

    I created python/buildmaster-config#228 to skip PGO builds on Python 3.7, since test_asyncio hangs on PGO builds.

    @vstinner
    Copy link
    Member Author

    New changeset 6790005 by Victor Stinner in branch 'master':
    bpo-43031: Set a timeout when running tests in PGO build (GH-24339)
    6790005

    @vstinner
    Copy link
    Member Author

    I close the issue.

    I added a timeout (20 min) in master when running tests for a PGO build.

    I created python/buildmaster-config#228 to skip PGO builds on Python 3.7, since test_asyncio hangs on PGO builds.

    Done. I remove all Python 3.7 PGO buildbot builders.

    The test_asyncio issue on Python 3.7 looks like a race condition which is likely already fixed in Python 3.8. Even if it's a bug, we cannot fix asyncio bugs in Python 3.7 anymore.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @elandorr elandorr mentioned this issue May 16, 2023
    Closed
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life tests Tests in the Lib/test dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @vstinner