Author brokenenglish
Recipients brokenenglish
Date 2017-01-23.18:58:25
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1485197905.85.0.319696153354.issue29353@psf.upfronthosting.co.za>
In-reply-to
Content
Hello.

I found a bug that causes incorrect handling of some values of "Content-Type" header. When you retrieve any URL with "Content-Type: message/rfc822" header, additional payload is added to HTTPMessage. In some cases it causes annoing warnings. Here is a code from packages requests and urllib3 that can cause warning output:
1. Checking and raising excpection in urllib3: https://github.com/shazow/urllib3/blob/0fb5e083b2adf7618db8c26e8e50206de09dd845/urllib3/util/response.py#L61-L66
2. The same code packaged in requests: https://github.com/kennethreitz/requests/blob/362da46e9a46da6e86e1907f03014384ab210151/requests/packages/urllib3/util/response.py#L61-L66
3. Logging the error to output: https://github.com/kennethreitz/requests/blob/362da46e9a46da6e86e1907f03014384ab210151/requests/packages/urllib3/connectionpool.py#L402-L407

The issue arises from the class email.feedparser.FeedParser that handles HTTP and email headers in the same way. So when it handles headers with the message with content-type "message/*" and some other, method _parsegen() tries to parse it like a message with another message (see comments to this condition: https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l295). If headers-only argument is set to True, we can avoid some redundant checks on HTTP headers (see this condition: https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l244).

I read the comment above (https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l241) but I am a newbie and I don't see any other solution to my problem for now. I can rewrite the patch if you show me a better way to fix this issue.

A patch with the unit test is atached.
History
Date User Action Args
2017-01-23 18:58:25brokenenglishsetrecipients: + brokenenglish
2017-01-23 18:58:25brokenenglishsetmessageid: <1485197905.85.0.319696153354.issue29353@psf.upfronthosting.co.za>
2017-01-23 18:58:25brokenenglishlinkissue29353 messages
2017-01-23 18:58:25brokenenglishcreate