classification
Title: Lazy import concurrent.futures.process and thread
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: cebtenzzre, levkivskyi, mark.dickinson, methane, pitrou, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2018-01-19 10:54 by methane, last changed 2021-10-15 13:33 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 5241 merged methane, 2018-01-19 11:00
PR 5316 merged methane, 2018-01-25 11:56
Messages (7)
msg310272 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-01-19 10:54
Since PEP 562 is implemented, we can use lazy imports easily.

Asyncio uses concurrent.futures.Future and concurrent.futures.ThreadPoolExecutor,
but not concurrent.futures.ProcessPoolExecutor by default.

Since importing concurrent.futures.process is slow, I want to import it lazily.
And it make sense to import concurrent.futures.thread too, because it's very
easy when we start to use __getattr__ once.

Here is quick benchmark:


$ ./python -m perf command ./python -c 'import asyncio'  # patched
.....................
command: Mean +- std dev: 60.5 ms +- 0.1 ms
$ git stash
$ ./python -m perf command ./python -c 'import asyncio'  # original
.....................
command: Mean +- std dev: 74.3 ms +- 0.2 ms
msg310310 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-01-20 00:54
New changeset 6690bb9f17d34eb3dec0aca8919d8d27d6c3c452 by INADA Naoki in branch 'master':
bpo-32596: Lazy import concurrent.futures.process and thread (GH-5241)
https://github.com/python/cpython/commit/6690bb9f17d34eb3dec0aca8919d8d27d6c3c452
msg310725 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-01-26 01:53
New changeset 4666ec597c38eea06a22bcfb4157d92a0abf891c by INADA Naoki in branch 'master':
bpo-32596: Make lazy-load portable (GH-5316)
https://github.com/python/cpython/commit/4666ec597c38eea06a22bcfb4157d92a0abf891c
msg388843 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2021-03-16 14:49
I don't think this was a good idea. Making some imports implicitly lazy introduces unpredictability in stdlib imports. Here is an example bug report:
https://issues.apache.org/jira/browse/ARROW-11983
msg403996 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-10-15 09:49
What are issues with using global variable as import target?
msg404012 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-10-15 13:05
See this comment.
https://github.com/python/cpython/pull/5241#discussion_r162765765
msg404016 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-10-15 13:33
Thank you Inada-san. My first intention when I seen this code was to simplify it, but then I have found that it was an intentional change.

It may be easy to detect such code and raise a syntax warning or error. Or get rid of this limitation and allow globals officially here.
History
Date User Action Args
2021-10-15 13:33:49serhiy.storchakasetmessages: + msg404016
2021-10-15 13:05:50methanesetmessages: + msg404012
2021-10-15 09:49:09serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg403996
2021-05-28 12:22:05mark.dickinsonsetnosy: + mark.dickinson
2021-04-22 17:51:36cebtenzzresetnosy: + cebtenzzre
2021-03-16 14:49:19pitrousetnosy: + pitrou
messages: + msg388843
2018-01-26 01:53:35methanesetmessages: + msg310725
2018-01-25 11:56:46methanesetpull_requests: + pull_request5162
2018-01-20 03:47:37methanesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-01-20 00:54:44methanesetmessages: + msg310310
2018-01-19 11:00:48methanesetkeywords: + patch
stage: patch review
pull_requests: + pull_request5087
2018-01-19 10:54:10methanecreate