Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lazy import concurrent.futures.process and thread #76777

Closed
methane opened this issue Jan 19, 2018 · 7 comments
Closed

Lazy import concurrent.futures.process and thread #76777

methane opened this issue Jan 19, 2018 · 7 comments
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@methane
Copy link
Member

methane commented Jan 19, 2018

BPO 32596
Nosy @mdickinson, @pitrou, @methane, @serhiy-storchaka, @1st1, @ilevkivskyi, @cebtenzzre
PRs
  • bpo-32596: Lazy import concurrent.futures.process and thread #5241
  • bpo-32596: Make lazy-load portable #5316
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-01-20.03:47:37.848>
created_at = <Date 2018-01-19.10:54:10.319>
labels = ['3.7', 'type-feature', 'library']
title = 'Lazy import concurrent.futures.process and thread'
updated_at = <Date 2021-10-15.13:33:49.212>
user = 'https://github.com/methane'

    bugs.python.org fields:

    activity = <Date 2021-10-15.13:33:49.212>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2018-01-20.03:47:37.848>
closer = 'methane'
components = ['Library (Lib)']
creation = <Date 2018-01-19.10:54:10.319>
creator = 'methane'
dependencies = []
files = []
hgrepos = []
issue_num = 32596
keywords = ['patch']
message_count = 7.0
messages = ['310272', '310310', '310725', '388843', '403996', '404012', '404016']
nosy_count = 7.0
nosy_names = ['mark.dickinson', 'pitrou', 'methane', 'serhiy.storchaka', 'yselivanov', 'levkivskyi', 'cebtenzzre']
pr_nums = ['5241', '5316']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue32596'
versions = ['Python 3.7']
    @methane
    Copy link
    Member Author

    methane commented Jan 19, 2018

    Since PEP-562 is implemented, we can use lazy imports easily.

    Asyncio uses concurrent.futures.Future and concurrent.futures.ThreadPoolExecutor,
    but not concurrent.futures.ProcessPoolExecutor by default.

    Since importing concurrent.futures.process is slow, I want to import it lazily.
    And it make sense to import concurrent.futures.thread too, because it's very
    easy when we start to use __getattr__ once.

    Here is quick benchmark:

    $ ./python -m perf command ./python -c 'import asyncio'  # patched
.....................
command: Mean +- std dev: 60.5 ms +- 0.1 ms
$ git stash
$ ./python -m perf command ./python -c 'import asyncio'  # original
.....................
command: Mean +- std dev: 74.3 ms +- 0.2 ms

    @methane methane added 3.7 (EOL) end of life stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Jan 19, 2018
    @methane
    Copy link
    Member Author

    methane commented Jan 20, 2018

    New changeset 6690bb9 by INADA Naoki in branch 'master':
    bpo-32596: Lazy import concurrent.futures.process and thread (GH-5241)
    6690bb9

    @methane methane closed this as completed Jan 20, 2018
    @methane
    Copy link
    Member Author

    methane commented Jan 26, 2018

    New changeset 4666ec5 by INADA Naoki in branch 'master':
    bpo-32596: Make lazy-load portable (GH-5316)
    4666ec5

    @pitrou
    Copy link
    Member

    pitrou commented Mar 16, 2021

    I don't think this was a good idea. Making some imports implicitly lazy introduces unpredictability in stdlib imports. Here is an example bug report:
    https://issues.apache.org/jira/browse/ARROW-11983

    @serhiy-storchaka
    Copy link
    Member

    What are issues with using global variable as import target?

    @methane
    Copy link
    Member Author

    methane commented Oct 15, 2021

    See this comment.
    #5241 (comment)

    @serhiy-storchaka
    Copy link
    Member

    Thank you Inada-san. My first intention when I seen this code was to simplify it, but then I have found that it was an intentional change.

    It may be easy to detect such code and raise a syntax warning or error. Or get rid of this limitation and allow globals officially here.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @methane @pitrou @serhiy-storchaka