This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Accelerate mimetypes.init on Windows
Type: performance Stage: resolved
Components: Windows Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: miss-islington, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2021-07-07 20:06 by steve.dower, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27059 merged steve.dower, 2021-07-07 20:20
PR 27071 merged miss-islington, 2021-07-08 15:48
Messages (6)
msg397110 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-07-07 20:06
mimetypes.init is slow on Windows because of the big registry search, which involves touching thousands of entries in order to add a few hundred into the initial table.

Things get even worse when audit hooks are enabled, because the winreg methods need to be hooked. However, we know that this particular operation is coming from core, and so we could skip these hooks.

Both issues can be trivially solved (helped) with a native accelerator function.
msg397113 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-07-07 20:40
It's only a rough benchmark, but my timings for `python -c "import mimetypes; mimetypes.init()` were ~200ms without the accelerator and ~100ms with the accelerator.

Since that includes all of startup, it's actually a better than 50% reduction for mimetypes.init(). I'm happy to take that without questioning precisely how much better it is ;)
msg397114 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-07-07 20:43
mimetypes doesn't _really_ support clearing, but `python.exe -m timeit "import mimetypes as M; M.types_map.clear(); M.init()"` is pretty close.

Without the accelerator: 133ms
With the accelerator: 27.8ms
msg397146 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-07-08 15:48
New changeset bbf2fb6c7ae78f40483606f467739a58cd747270 by Steve Dower in branch 'main':
bpo-44582: Accelerate mimetypes.init on Windows with a native accelerator (GH-27059)
https://github.com/python/cpython/commit/bbf2fb6c7ae78f40483606f467739a58cd747270
msg397147 - (view) Author: miss-islington (miss-islington) Date: 2021-07-08 16:13
New changeset 08697ac5d1543fca3629c719ab43e50d73021631 by Miss Islington (bot) in branch '3.10':
bpo-44582: Accelerate mimetypes.init on Windows with a native accelerator (GH-27059)
https://github.com/python/cpython/commit/08697ac5d1543fca3629c719ab43e50d73021631
msg397157 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-07-08 17:03
Dropping the 3.9 backport. Pretty sure it only needs a clinic regen, but it's also not worth the risk of changed behaviour.

We could consider it again after 3.10 has shipped if people really want it (and can help validate it).
History
Date User Action Args
2022-04-11 14:59:47adminsetgithub: 88748
2021-07-08 17:03:31steve.dowersetstatus: open -> closed
versions: - Python 3.9
messages: + msg397157

resolution: fixed
stage: patch review -> resolved
2021-07-08 16:13:14miss-islingtonsetmessages: + msg397147
2021-07-08 15:48:53miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25622
2021-07-08 15:48:51steve.dowersetmessages: + msg397146
2021-07-07 20:43:16steve.dowersetmessages: + msg397114
2021-07-07 20:40:12steve.dowersetmessages: + msg397113
2021-07-07 20:20:45steve.dowersetnosy: + paul.moore, tim.golden, zach.ware
components: + Windows
2021-07-07 20:20:21steve.dowersetkeywords: + patch
stage: patch review
pull_requests: + pull_request25614
2021-07-07 20:06:39steve.dowercreate