classification
Title: cgi.parse_multipart() requires undocumented CONTENT-LENGTH in Python 3.7
Type: behavior Stage: resolved
Components: Documentation, Library (Lib) Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: orsenthil Nosy List: Tercio Gaudencio Filho, docs@python, fdrake, martin.panter, miss-islington, ned.deily, orsenthil, quentel, rogerduran, xtreak, yan12125, ytvwld
Priority: normal Keywords: 3.7regression, patch

Created on 2018-07-25 14:55 by yan12125, last changed 2020-06-15 17:59 by orsenthil. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8530 merged python-dev, 2018-07-28 15:07
PR 20890 merged miss-islington, 2020-06-15 14:59
PR 20891 merged miss-islington, 2020-06-15 14:59
PR 20892 merged miss-islington, 2020-06-15 14:59
Messages (11)
msg322362 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2018-07-25 14:55
In Python 3.7, cgi.parse_multipart() requires "CONTENT-LENGTH" in the second parameter `pdict`. This is not necesary in Python 3.6.

For example, the following code runs with 3.6:

$ python3.6
Python 3.6.6 (default, Jun 27 2018, 13:11:40) 
[GCC 8.1.1 20180531] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cgi, io
>>> data = b'--heyDavid\r\nContent-Disposition: form-data; name="cfield"\r\n\r\njust a string\r\n\r\n--heyDavid--\r\n'
>>> cgi.parse_multipart(io.BytesIO(data), {"boundary": b"heyDavid"})
{'cfield': [b'just a string\r\n']}

While not on 3.7:

$ python3.7
Python 3.7.0 (default, Jul 15 2018, 10:44:58) 
[GCC 8.1.1 20180531] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cgi, io
>>> data = b'--heyDavid\r\nContent-Disposition: form-data; name="cfield"\r\n\r\njust a string\r\n\r\n--heyDavid--\r\n'
>>> cgi.parse_multipart(io.BytesIO(data), {"boundary": b"heyDavid"})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/cgi.py", line 220, in parse_multipart
    headers['Content-Length'] = pdict['CONTENT-LENGTH']
KeyError: 'CONTENT-LENGTH'

I looked into the source code of CPython, and found CONTENT-LENGTH in nowhere but cgi.py and test_cgi.py. I guess it has the same meaning as the Content-Length header in HTTP or CONTENT_LENGTH environment variable in CGI? It would be great to document such a backward-incompatible behavior.

Environment: Arch Linux with Python 3.6.6 from [extra] repo and 3.7.0 from [staging] repo.

CC the author and the reviewer of issue29979, where relevant codes are introduced.
msg322541 - (view) Author: roger (rogerduran) * Date: 2018-07-28 10:55
working on this on europython
msg353545 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2019-09-30 06:45
Looks like the change causing this is revision cc3fa20. I would remove the reference to pdict['CONTENT-LENGTH'].
msg371340 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-06-12 08:32
I note this is marked as a 3.7regression and still open. Since the cutoff for the final 3.7 bugfix mode release is in a few days, I'm assuming this means that 3.7 users will have to live with this regression.  If you feel that is a problem, speak up now.
msg371350 - (view) Author: Niklas Sombert (ytvwld) Date: 2020-06-12 10:33
Since there's the easy workaround of just setting pdict["CONTENT-LENGTH"] I personally don't have a problem with this staying the way it is.

But it would be really nice if this were documented somewhere.
msg371367 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2020-06-12 13:27
I will spend time on this on 12-June or 13-June, 2020.
At the very least, if no changes are required, I will document this for 3.7 and higher versions.
msg371557 - (view) Author: miss-islington (miss-islington) Date: 2020-06-15 14:59
New changeset d8cf3514dd4682419a66f6e834bb384ee34afc95 by roger in branch 'master':
bpo-34226: fix cgi.parse_multipart without content_length (GH-8530)
https://github.com/python/cpython/commit/d8cf3514dd4682419a66f6e834bb384ee34afc95
msg371563 - (view) Author: miss-islington (miss-islington) Date: 2020-06-15 15:33
New changeset aa83935a56d1fd4d72d4de5f0278a240a2d6844d by Miss Islington (bot) in branch '3.7':
[3.7] bpo-34226: fix cgi.parse_multipart without content_length (GH-8530) (GH-20892)
https://github.com/python/cpython/commit/aa83935a56d1fd4d72d4de5f0278a240a2d6844d
msg371565 - (view) Author: miss-islington (miss-islington) Date: 2020-06-15 15:51
New changeset b87453f94fd391e6700eb25dd91de2e56aeeb98f by Miss Islington (bot) in branch '3.9':
bpo-34226: fix cgi.parse_multipart without content_length (GH-8530)
https://github.com/python/cpython/commit/b87453f94fd391e6700eb25dd91de2e56aeeb98f
msg371566 - (view) Author: miss-islington (miss-islington) Date: 2020-06-15 15:51
New changeset c72b7f703eec45c1f4006bf5f65092daedaec46e by Miss Islington (bot) in branch '3.8':
bpo-34226: fix cgi.parse_multipart without content_length (GH-8530)
https://github.com/python/cpython/commit/c72b7f703eec45c1f4006bf5f65092daedaec46e
msg371583 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2020-06-15 17:59
This is now fixed in all active versions of python. No documentation changes required for this change.
History
Date User Action Args
2020-06-15 17:59:28orsenthilsetmessages: + msg371583
2020-06-15 16:04:52orsenthilsetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.9, Python 3.10
2020-06-15 15:51:39miss-islingtonsetmessages: + msg371566
2020-06-15 15:51:37miss-islingtonsetmessages: + msg371565
2020-06-15 15:33:39miss-islingtonsetmessages: + msg371563
2020-06-15 14:59:24miss-islingtonsetpull_requests: + pull_request20075
2020-06-15 14:59:16miss-islingtonsetpull_requests: + pull_request20074
2020-06-15 14:59:08miss-islingtonsetpull_requests: + pull_request20073
2020-06-15 14:59:02miss-islingtonsetnosy: + miss-islington
messages: + msg371557
2020-06-12 13:27:16orsenthilsetassignee: docs@python -> orsenthil
messages: + msg371367
2020-06-12 10:33:47ytvwldsetmessages: + msg371350
2020-06-12 08:32:17ned.deilysetnosy: + ned.deily
messages: + msg371340
2019-09-30 06:45:49martin.pantersetkeywords: + 3.7regression
nosy: + martin.panter
messages: + msg353545

2019-09-03 14:00:53ytvwldsetnosy: + ytvwld
2019-07-12 18:55:03fdrakesetnosy: + fdrake
2018-08-30 18:58:24Tercio Gaudencio Filhosetnosy: + Tercio Gaudencio Filho
2018-07-28 15:07:11python-devsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8047
2018-07-28 10:55:08rogerduransetnosy: + rogerduran
messages: + msg322541
2018-07-25 15:19:20xtreaksetnosy: + xtreak
2018-07-25 14:55:40yan12125create