Title: Use bytearray in urllib.unquote_to_bytes
Type: performance Stage: patch review
Components: Extension Modules Versions: Python 3.11, Python 3.10, Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eng.mustafaelagamey, gregory.p.smith, orsenthil, terry.reedy
Priority: normal Keywords: patch

Created on 2021-06-07 15:07 by eng.mustafaelagamey, last changed 2021-06-07 20:09 by gregory.p.smith.

Pull Requests
URL Status Linked Edit
PR 26576 closed eng.mustafaelagamey, 2021-06-07 15:16
Messages (2)
msg395280 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-06-07 19:42
'eng' claimed in original title that "urllib.parse.parse_qsl cannot parse large data".  On original PR, said problem with 6-7 millions bytes.

Claim should be backed up by a generated example that fails with original code and succeeds with new code.  Claims of 'faster' also needs some examples.

Original PRs must nearly all propose merging a branch created from main into main.  Performance enhancements are often not backported.
msg395281 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2021-06-07 20:09
fwiw this sort of thing may be reasonable to backport to 3.9 as it is more than just a performance enhancement but also a resource consumption bug and should result in no behavior change.

In case of form contain very large data ( in my case the string to parse was about 6000000 byte )
Old code use list of bytes during parsing consumes a lot of memory
New code will use bytearry , which use less memory
""" - text from the original PR
Date User Action Args
2021-06-07 20:09:11gregory.p.smithsetversions: + Python 3.9, Python 3.10
2021-06-07 20:09:04gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg395281
2021-06-07 19:42:17terry.reedysetnosy: + terry.reedy, orsenthil
title: urllib.parse.parse_qsl cannot parse large data -> Use bytearray in urllib.unquote_to_bytes
messages: + msg395280

versions: + Python 3.11, - Python 3.8
2021-06-07 15:16:13eng.mustafaelagameysetkeywords: + patch
stage: patch review
pull_requests: + pull_request25164
2021-06-07 15:09:38eng.mustafaelagameysettitle: urllib cannot parse large data -> urllib.parse.parse_qsl cannot parse large data
2021-06-07 15:07:38eng.mustafaelagameycreate