Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove support for threads-less builds #75551

Closed
pitrou opened this issue Sep 6, 2017 · 29 comments
Closed

Remove support for threads-less builds #75551

pitrou opened this issue Sep 6, 2017 · 29 comments
Labels
3.7 (EOL) end of life build The build process and cross-build interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@pitrou
Copy link
Member

pitrou commented Sep 6, 2017

BPO 31370
Nosy @brettcannon, @gpshead, @pitrou, @tiran, @bitdancer, @briancurtin, @pmp-p, @serhiy-storchaka, @kb-1000
PRs
  • bpo-31370: Remove support for threads-less builds #3385
  • bpo-31370: Simplify run_tests.py #3482
  • Restore dummy_threading and _dummy_thread, but deprecate them (bpo-31370) #3648
  • Trivial cleanups following bpo-31370 #3649
  • bpo-31370: Remove references to threadless builds #8805
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2017-11-03.06:42:40.326>
    created_at = <Date 2017-09-06.21:10:49.815>
    labels = ['interpreter-core', 'type-feature', '3.7', 'build']
    title = 'Remove support for threads-less builds'
    updated_at = <Date 2022-03-09.09:35:58.994>
    user = 'https://github.com/pitrou'

    bugs.python.org fields:

    activity = <Date 2022-03-09.09:35:58.994>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2017-11-03.06:42:40.326>
    closer = 'berker.peksag'
    components = ['Build', 'Interpreter Core']
    creation = <Date 2017-09-06.21:10:49.815>
    creator = 'pitrou'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 31370
    keywords = ['patch']
    message_count = 29.0
    messages = ['301518', '301520', '301521', '301539', '301543', '301559', '301595', '301596', '301805', '301964', '301968', '301971', '301973', '301980', '301987', '301988', '302474', '302476', '302484', '302492', '326681', '348490', '408243', '408257', '408501', '408508', '408518', '408575', '408594']
    nosy_count = 11.0
    nosy_names = ['brett.cannon', 'gregory.p.smith', 'pitrou', 'christian.heimes', 'Arfrever', 'r.david.murray', 'brian.curtin', 'pmpp', 'serhiy.storchaka', 'kb1000', 'Steve Gomez']
    pr_nums = ['3385', '3482', '3648', '3649', '8805']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue31370'
    versions = ['Python 3.7']

    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 6, 2017

    The --without-threads option has a lot of repercussions on the code base, forcing us to maintain specific paths for builds with threads disabled, for no useful purpose.

    @pitrou pitrou added 3.7 (EOL) end of life build The build process and cross-build interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement labels Sep 6, 2017
    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 6, 2017

    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 6, 2017

    Question: should we keep Modules/Setup.config.in? It's going to become empty:
    https://github.com/python/cpython/blob/master/Modules/Setup.config.in

    @vstinner
    Copy link
    Member

    vstinner commented Sep 6, 2017

    The PEP-11 should be updated just to mention that platforms without threading support are not supported in Python 3.7.

    @vstinner
    Copy link
    Member

    vstinner commented Sep 6, 2017

    We had a buildbot testing --without-threads, but it's gone. I just removed its unused class from the buildbot configuration:

    python/buildmaster-config@091f52a

    @gpshead
    Copy link
    Member

    gpshead commented Sep 7, 2017

    I guess Setup.config.in can go. It's only purpose was for autoconf driven Setup changes.

    @vstinner
    Copy link
    Member

    vstinner commented Sep 7, 2017

    New changeset a6a4dc8 by Victor Stinner (Antoine Pitrou) in branch 'master':
    bpo-31370: Remove support for threads-less builds (bpo-3385)
    a6a4dc8

    @vstinner
    Copy link
    Member

    vstinner commented Sep 7, 2017

    PEP-11 was updated, the change was merged, this issue can now be closed. Thanks Antoine!

    @vstinner vstinner closed this as completed Sep 7, 2017
    @Arfrever
    Copy link
    Mannequin

    Arfrever mannequin commented Sep 10, 2017

    Also 'import threading' code in Tools/scripts/run_tests.py should be cleaned.

    @Arfrever
    Copy link
    Mannequin

    Arfrever mannequin commented Sep 12, 2017

    Lib/_dummy_thread.py and Lib/dummy_threading.py were deleted, but Doc/library/_dummy_thread.rst and Doc/library/dummy_threading.rst still exist.

    Probably Doc/whatsnew/3.7.rst should mention that these modules were deleted.

    Or maybe they should be restored and firstly deprecated, before deletion in a later version of Python.

    @Arfrever Arfrever mannequin reopened this Sep 12, 2017
    @vstinner
    Copy link
    Member

    Arfrever: "Lib/_dummy_thread.py and Lib/dummy_threading.py were deleted, but Doc/library/_dummy_thread.rst and Doc/library/dummy_threading.rst still exist."

    Ah, it should be deleted as well.

    Arfrever: "Probably Doc/whatsnew/3.7.rst should mention that these modules were deleted."

    Right.

    Arfrever: "Or maybe they should be restored and firstly deprecated, before deletion in a later version of Python."

    I don't think that it's worth it, but I suggest to add a warning to Python 3.6 documentation. Something like the following warning in Python 2.7 documentation of the commands module:
    "Deprecated since version 2.6: The commands module has been removed in Python 3. Use the subprocess module instead."
    https://docs.python.org/2/library/commands.html

    @vstinner
    Copy link
    Member

    Oh, Arfrever wrote me on IRC that the removal of dummy_threading broke portage on Gentoo: https://bugs.gentoo.org/630730

    eventlet also uses dummy_threading:

    eventlet/support/dns/entropy.py:23: import dummy_threading as _threading
    eventlet/support/dns/resolver.py:29: import dummy_threading as _threading

    Hum, maybe we need one release with the deprecated module, and only remove it in Python 3.8?

    @tiran
    Copy link
    Member

    tiran commented Sep 12, 2017

    +1 to restore dummy_threading and go through a proper deprecation phase in 3.7+.

    -0 to restore _dummy_thread. It's an internal, private module.

    @bitdancer
    Copy link
    Member

    dummy_threading should definitely not have been removed, and like all the other APIs should not be removed until 2.7 is dead. Deprecating it is of course fine :)

    @bitdancer
    Copy link
    Member

    And actually, I wouldn't be surprised if eventlet depended on the *functionality* in _dummy_threading, so you probably need to restore that, too.

    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 18, 2017

    eventlet also uses dummy_threading

    Not sure what you mean by that. eventlet uses the regular try/except import dance around "import threading":

    https://github.com/eventlet/eventlet/blob/master/eventlet/support/dns/entropy.py#L20-L23
    https://github.com/eventlet/eventlet/blob/master/eventlet/support/dns/resolver.py#L23-L26

    As for portage, Arfrever fixed it :-)

    https://gitweb.gentoo.org/proj/portage.git/commit/?id=504f66b0e25281e4465ebeceb799c3e54ff2b884

    That said, I am not against bringing dummy_threading back, though it would sound rather weird to use it unconditionally...

    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 18, 2017

    #3648 restores dummy_threading and _dummy_thread, but deprecates them.

    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 18, 2017

    New changeset b43c4ca by Antoine Pitrou in branch 'master':
    Restore dummy_threading and _dummy_thread, but deprecate them (bpo-31370) (bpo-3648)
    b43c4ca

    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 18, 2017

    New changeset 88c60c9 by Antoine Pitrou in branch 'master':
    Trivial cleanups following bpo-31370 (bpo-3649)
    88c60c9

    @briancurtin
    Copy link
    Member

    New changeset eef0596 by Brian Curtin (Zackery Spytz) in branch 'master':
    bpo-31370: Remove references to threadless builds (bpo-8805)
    eef0596

    @kb-1000
    Copy link
    Mannequin

    kb-1000 mannequin commented Jul 26, 2019

    Unfortunately, this also leads to less good LTO if you use a single static executable, similar to what Tools/freeze produces but including libpython itself. I'm currently trying to make LLVM do that, but since I can't build without threads, it'd be extremely hard to tell it that it can remove all threading related things (like TLS) and inline them. This means I'll have to patch CPython on my own to remove thread support, but unfortunately this also means I can't guarantee that it continues to work, as well as that it makes it a lot more likely that my build won't support many third-party libraries.

    @brettcannon
    Copy link
    Member

    This has unfortunately turned out to be a blocker on getting WASI support as there's not direct threading support in WebAssembly.

    @gpshead
    Copy link
    Member

    gpshead commented Dec 10, 2021

    https://web.dev/webassembly-threads/ suggests otherwise.

    @brettcannon
    Copy link
    Member

    https://web.dev/webassembly-threads/ suggests otherwise.

    It actually doesn't as that post isn't for WASI, it's for WASM in the browser (I chose my acronyms carefully 😉).

    WASI is like POSIX for WebAssembly, so it's meant for desktop usage and thus no browser or JS engine is involved to provide web workers.

    @vstinner
    Copy link
    Member

    This has unfortunately turned out to be a blocker on getting WASI support as there's not direct threading support in WebAssembly.

    This issue is now closed and unrelated to WASI support. Can you please open a new separated issue for it?

    @serhiy-storchaka
    Copy link
    Member

    Concur with Victor. It is fine to add a link to other discussion in a closed issue, but an issue closed many years ago is not a good place for a new discussion at all.

    @brettcannon
    Copy link
    Member

    I am not opening a new discussion; this is just recording this fact here as it has come up in other places on the internet.

    @vstinner
    Copy link
    Member

    Either reopen the issue or open a new issue. Only people subscribed to this bug are aware of the recent comments. Closed issues are hidden from the bugs home page and from the default search.

    If you consider that Python must again support thread-less platforms, IMO it's a new feature and a new issue must be opened.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life build The build process and cross-build interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    9 participants