classification
Title: Use functools.lru_cache in urllib.parse instead of 1996 custom caching
Type: enhancement Stage: commit review
Components: Library (Lib) Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: gregory.p.smith
Priority: normal Keywords: patch

Created on 2021-05-01 18:17 by gregory.p.smith, last changed 2021-05-12 00:02 by gregory.p.smith. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 25798 merged gregory.p.smith, 2021-05-01 22:50
Messages (3)
msg392614 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2021-05-01 18:17
`urllib.parse` has custom caching code for both `urlsplit()` and `quote()`.  From 1996.

https://github.com/python/cpython/commit/3fd32ecd9232fcb041b9f1f7a19a1e7e65cf11a0
https://github.com/python/cpython/commit/74495409861b357d9925937d6576229c74e2550d

with a truthful comment added by Nick in 2010 that we should just use functools.lru_cache.

https://github.com/python/cpython/commit/9fc443cf590c76d4b979c46dc954d3956cee0319#diff-b3712475a413ec972134c0260c8f1eb1deefb66184f740ef00c37b4487ef873e

time to clean up this cruft and do that.

I'm waiting for after the 3.10 cut and a still in progress urllib.parse security fix to land before rebasing my soon to be attached PR to avoid code conflicts.
msg392619 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-05-01 19:05
While you're cleaning up the module, take a look at the Quoter class.  It overrides __init__ and __missing__, so Quoter is not using any of the defaultdict features at all.  I'm thinking it could just inherit from dict.
msg392620 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2021-05-01 19:23
Yeah, the Quoter class seems a little odd...

Past notes of some the caching performance around the character quoting for quote() can be found in https://bugs.python.org/issue1285086 circa 2005-2010.
History
Date User Action Args
2021-05-12 00:02:19gregory.p.smithsetstatus: open -> closed
resolution: fixed
stage: patch review -> commit review
2021-05-01 22:50:47gregory.p.smithsetkeywords: + patch
stage: patch review
pull_requests: + pull_request24486
2021-05-01 19:23:36gregory.p.smithsetnosy: - rhettinger
messages: + msg392620
2021-05-01 19:05:59rhettingersetnosy: + rhettinger
messages: + msg392619
2021-05-01 18:17:15gregory.p.smithcreate