-
-
Notifications
You must be signed in to change notification settings - Fork 29.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cgi.FieldStorage can't parse multipart part headers with Content-Length and no filename in Content-Disposition #68952
Comments
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.49)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cgi
>>> from io import BytesIO
>>>
>>> BOUNDARY = "JfISa01"
>>> POSTDATA = """--JfISa01
... Content-Disposition: form-data; name="submit-name"
... Content-Length: 5
...
... Larry
... --JfISa01"""
>>> env = {
... 'REQUEST_METHOD': 'POST',
... 'CONTENT_TYPE': 'multipart/form-data; boundary={}'.format(BOUNDARY),
... 'CONTENT_LENGTH': str(len(POSTDATA))}
>>> fp = BytesIO(POSTDATA.encode('latin-1'))
>>> fs = cgi.FieldStorage(fp, environ=env, encoding="latin-1")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 571, in __init__
self.read_multi(environ, keep_blank_values, strict_parsing)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 726, in read_multi
self.encoding, self.errors)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 573, in __init__
self.read_single()
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 736, in read_single
self.read_binary()
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 758, in read_binary
self.file.write(data)
TypeError: must be str, not bytes
>>> This happens because of a mismatch between the code that creates a temp file to write to and the code that chooses to read in binary mode or not:
When I've reviewed the relevant RFCs, and I'm not really sure what the correct way to handle this is. I don't believe At the very least, I think this behavior is confusing and unexpected. Some libraries, like Retrofit2, will by default include I've made an attempt to work in the way I'd expect, and attached a patch, but I'm really not sure if it's the proper decision. My patch kind of naively accepts the existing semantics of |
I realized my formatting was poor, making it hard to quickly test the issue. Here's a cleaner version: import cgi
from io import BytesIO
BOUNDARY = "JfISa01"
POSTDATA = """--JfISa01
Content-Disposition: form-data; name="submit-name"
Content-Length: 5
|
Yes, I will be able to review the patch next week 2015-07-31 18:13 GMT+02:00 STINNER Victor <report@bugs.python.org>:
|
I don't really see why there is a Content-Length in the headers of a Content-Type: multipart/form-data; boundary=AaB03x Larry In case a user agent would insert it, I think the best would be to headers = parser.close() add these lines : if 'content-length' in headers:
del headers['content-length'] It's hard to see the potential side effects but I think it's cleaner Peter, does this make sense ? If so, can you submit another patch ? |
Yeah, I think that makes the most sense to me as well. I tried to make a minimum-impact patch, but this feels cleaner. If we remove the Content-Length header, the I'll re-work the patch and make sure the tests I added still add value. Thanks! |
A new patch that simply removes Content-Length from part headers when present. |
Victor, you can apply the patch and close the issue.
|
Not today. I'm in holiday. Ping me in two weeks or find another core dev. |
Hi Victor, |
Pradeep, that error seems to be in Barbican. This bug and patch only addresses content-length headers in MIME multipart messages. |
New changeset 11e9f34169d1 by Victor Stinner in branch '3.4': New changeset 5b9209e4c3e4 by Victor Stinner in branch '3.5': New changeset 0ff1acc89cf0 by Victor Stinner in branch 'default': |
I applied the fix, thanks Peter for the report and the fix, thanks Pierre for the review.
I don't know, but you can try to apply the patch locally if you want, or download the Python 3.4 using Mercurial. |
I've uploaded a patchset to bug bpo-27777 that fixes this issue by fixing make_file, and doesn't cause Python to throw out the content-length information. It also fixes FieldStorage for when you provide it a non-multipart form submission and there is no list in FS. Please see http://bugs.python.org/issue27777 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: