This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Remove shlwapi dependency on Windows
Type: performance Stage: resolved
Components: Windows Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: eryksun, miss-islington, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2021-11-05 01:04 by steve.dower, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29417 merged steve.dower, 2021-11-05 01:06
PR 29435 merged steve.dower, 2021-11-05 23:12
PR 29436 merged miss-islington, 2021-11-06 00:25
Messages (6)
msg405762 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-05 01:04
According to https://twitter.com/BruceDawson0xB/status/1455714820485894151?s=20 there are some serious performance implications from referencing shlwapi.dll.

It turns out, we only use it for one native path calculation function, which is easily replaceable (since Windows 8). So we can drop the dependency and get a startup (and shutdown) improvement back to 3.9.
msg405763 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-05 01:28
FTR, I see about 1-1.5ms improvement (using timeit to do a check_call) out of ~32ms total startup time.

I also don't actually see gdi32 being transitively loaded as claimed in the Twitter thread, even back to 3.8. So presumably there's something else going on to cause that issue.

Either way, this is an easy change with non-zero benefit (at the very least, it gets rid of a dependency we are better off avoiding these days).
msg405831 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-11-05 22:30
> I also don't actually see gdi32 being transitively loaded as
> claimed in the Twitter thread, even back to 3.8. So presumably
> there's something else going on to cause that issue.

Since Windows XP, shlwapi.dll has increasingly made use of delay loading, but up to Windows 8.1 it was still immediately loading user32.dll and gdi32.dll. In Windows 10, shlwapi.dll finally made them delay loaded dependencies. Also, in Windows 10, PathIsRelativeW is forwarded to the implementation in kernelbase.dll, not that it matters from the end user's perspective since it's not in an API set.

Regarding user32.dll, there's still an issue with ssl, hashlib, winsound, and ctypes not delay loading it, or DLLs that directly depend on it such as ole32.dll and oleaut32.dll. In most use cases these DLLs would never be loaded. 

When user32.dll loads, the process and the loading thread are converted to GUI versions, i.e. they're extended in the kernel with the window manager's Win32Process and Win32Thread structures, and the process connects to a window station and desktop (e.g. r"WinSta0\Default"). One concern with converting to a GUI process is that the system doesn't send CTRL_LOGOFF_EVENT and CTRL_SHUTDOWN_EVENT console events to GUI processes. Thus a console script can't handle those events with a simple ctypes-based handler as long as importing ctypes always loads user32.dll.
msg405833 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-05 23:06
New changeset a4774f42e35861c4bb16928cffb011c2d8a285ac by Steve Dower in branch 'main':
bpo-45720: Drop references to shlwapi.dll on Windows (GH-29417)
https://github.com/python/cpython/commit/a4774f42e35861c4bb16928cffb011c2d8a285ac
msg405836 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-06 00:25
New changeset 804ea41211b042fa20c3cd8c0457bbfa3873128a by Steve Dower in branch '3.10':
bpo-45720: Drop references to shlwapi.dll on Windows (GH-29417)
https://github.com/python/cpython/commit/804ea41211b042fa20c3cd8c0457bbfa3873128a
msg405840 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-06 01:39
New changeset 5017306c8732b3ceda878db13088f8c2cf0c5e71 by Miss Islington (bot) in branch '3.9':
bpo-45720: Drop references to shlwapi.dll on Windows (GH-29417)
https://github.com/python/cpython/commit/5017306c8732b3ceda878db13088f8c2cf0c5e71
History
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 89883
2021-11-06 01:39:50steve.dowersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-11-06 01:39:02steve.dowersetmessages: + msg405840
2021-11-06 00:25:38steve.dowersetmessages: + msg405836
2021-11-06 00:25:37miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27690
2021-11-05 23:12:30steve.dowersetpull_requests: + pull_request27689
2021-11-05 23:06:49steve.dowersetmessages: + msg405833
2021-11-05 22:30:55eryksunsetmessages: + msg405831
2021-11-05 01:28:55steve.dowersetmessages: + msg405763
2021-11-05 01:07:24steve.dowersetnosy: + eryksun
2021-11-05 01:06:26steve.dowersetkeywords: + patch
stage: patch review
pull_requests: + pull_request27670
2021-11-05 01:04:25steve.dowercreate