This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multipart/related header causes false positive StartBoundaryNotFoundDefect and MultipartInvariantViolationDefect
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, cschmidbauer, martin.panter, maxking, r.david.murray, tzickel
Priority: normal Keywords: patch

Created on 2019-03-07 15:31 by cschmidbauer, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 12214 open python-dev, 2019-03-07 16:09
Repositories containing patches
https://github.com/genesiscloud/cpython/tree/fix-issue-36226
Messages (5)
msg337395 - (view) Author: Christian Schmidbauer (cschmidbauer) * Date: 2019-03-07 15:31
The current implementation of `multipart/related` in urllib triggers header defects even though the headers are valid:
`[StartBoundaryNotFoundDefect(), MultipartInvariantViolationDefect()]`

The example header is valid according to RFC 2387 (https://tools.ietf.org/html/rfc2387):
```
Content-Type: multipart/related; boundary="==="

```

Both defects are triggered by the fact that httplib only passes on headers to the underlying email parser, while the email parser assumes to receive a full message. The simple fix is to tell the underlying email parser that we are only passing the header: 0a89fc15c93c271eb08e46e2cda9a72adb97d633

The second issue is related, but independent: The underlying email parser checks if the parsed message is of type multipart by checking of the object "root" is of type list. As we only passed the header (and set `headersonly=True`), the check does makes no sense anymore at this point, creating a false positive: fdc7c47b77e330a36255fd00dc36accd72824e5b
msg337401 - (view) Author: Christian Schmidbauer (cschmidbauer) * Date: 2019-03-07 16:13
Apologies, here are the correct commit IDs:

https://github.com/python/cpython/commit/89285439c7f94a3e62cee3d15e343218903c2af8

https://github.com/python/cpython/pull/12214/commits/a82e662ab3339072d7b86a8090989fba60ef9c37
msg337444 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2019-03-07 22:13
Probably the same as Issue 29353. I remember than enabling "headersonly" can create inconsistencies in the message object. But I don't remember the details.

According to Issue 29991 (another duplicate), my patch for Issue 24363 might help. But I don't think I got much enthusiasm reviewing that (and I don't have time to spend on it now).
msg337661 - (view) Author: Christian Schmidbauer (cschmidbauer) * Date: 2019-03-11 12:37
@martin.panter: I see a relation to issue 29353, but I don't see why this report here is a duplicate. Could you elaborate on this?

Issue 29991 contains parts of what I reported here, but it is closed "resolved" and refers back to 29353.

I also tried your patch "policy-flag.patch" and it did not help in the regard of the bug here and tests which are included in the PR.
msg351114 - (view) Author: (tzickel) * Date: 2019-09-04 04:23
It should be noted that this causes a big headache for users of requests / urllib3 / etc... as those print on each multipart response a logging warning based on this bug, and it might cause people to go try debugging valid code:

https://github.com/urllib3/urllib3/issues/800

https://github.com/psf/requests/issues/3001

https://github.com/diyan/pywinrm/issues/269

https://github.com/jborean93/pypsrp/issues/39

https://github.com/home-assistant/home-assistant/pull/17042

https://github.com/Azure/azure-storage-python/issues/167

and others....
History
Date User Action Args
2022-04-11 14:59:12adminsetgithub: 80407
2019-09-04 15:58:20ned.deilysetnosy: + barry, r.david.murray, maxking
2019-09-04 04:23:24tzickelsetnosy: + tzickel
messages: + msg351114
2019-03-11 12:37:23cschmidbauersetmessages: + msg337661
2019-03-07 22:13:38martin.pantersetnosy: + martin.panter
messages: + msg337444
2019-03-07 16:13:44cschmidbauersetmessages: + msg337401
2019-03-07 16:09:12python-devsetkeywords: + patch
stage: patch review
pull_requests: + pull_request12204
2019-03-07 15:33:36cschmidbauersethgrepos: + hgrepo381
2019-03-07 15:31:17cschmidbauercreate