classification
Title: Add skip_host and skip_accept_encoding to HTTPConnection.request()
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.10, Python 3.9
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: iritkatriel, kanzure, martin.panter, orsenthil, pitrou
Priority: normal Keywords: patch

Created on 2012-12-31 23:57 by kanzure, last changed 2020-11-08 15:50 by kanzure. This issue is now closed.

Files
File name Uploaded Description Edit
httplib_better_header_skips.patch kanzure, 2012-12-31 23:57 add skip_accept_encoding and skip_host to HttpConnection.request review
Messages (7)
msg178719 - (view) Author: Bryan Bishop (kanzure) Date: 2012-12-31 23:57
Sometimes I am using httplib/http.client and the server is not exactly conforming to HTTP specs. I need to be able to specify the exact headers that are sent to the server. By default, httplib/http.client injects headers like "Host" and "Accept-Encoding".

Issue #831747 added skip_accept_encoding to httplib's putrequest method, but not on the request method. This current patch exposes these two toggles on the request method. This way, headers can be controlled without manually calling the connect/send/endheaders methods.

As a result, urllib/urllib3 can call urlopen and pass in these attributes to more directly control the headers sent to the remote server.
msg178730 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-01 11:40
I'm not sure this is a desirable feature in the stdlib, but regardless, your solution isn't very scalable: a new argument will have to be added each time someone wants to avoid sending a given header.

Another possibility would be to allow passing None in values of the `headers` dict, in which case the given header wouldn't be send at all.
msg178741 - (view) Author: Bryan Bishop (kanzure) Date: 2013-01-01 18:30
On Tue, Jan 1, 2013 at 5:41 AM, Antoine Pitrou wrote:
> Another possibility would be to allow passing None in values of the `headers` dict, in which case the given header wouldn't be send at all.

I agree that your solution is more scaling-friendly than the patch I originally submitted. But from another perspective, consider how alien it is to have to explicitly add keys set to None in the headers to get desired behavior.

Maybe there could be a single flag "strict" (or possibly a better name) that would indicate that the dict is in fact what I want to send and that it should not be modified? Or perhaps there should be "base_headers" list (not a dictionary) of header keys of which defaults to not inject?

Also, would making {"Accept-Encoding": None} send no "Accept-Encoding" header be backwards incompatible with the current behavior?
msg178743 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-01 18:40
> On Tue, Jan 1, 2013 at 5:41 AM, Antoine Pitrou wrote:
> > Another possibility would be to allow passing None in values of the
> `headers` dict, in which case the given header wouldn't be send at
> all.
> 
> I agree that your solution is more scaling-friendly than the patch I
> originally submitted. But from another perspective, consider how alien
> it is to have to explicitly add keys set to None in the headers to get
> desired behavior.

Well, that is not a common need, is it?
That said, we could indeed have a separate argument "omit_headers"
listing the headers that we don't want to send.
msg245790 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-06-25 02:57
I tend to think that this proposal should be rejected. If you need low-level control over the header fields, just use putrequest(), putheader() and endheaders() methods instead. And adding these flags to request() isn’t going to help with urlopen(), unless the Request class was extended in a similar way.
msg380489 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-11-07 02:38
Bryan, as there isn't unanimous agreement about the feature you suggested: if you are still interested in working on this, could you bring it up on python-ideas@python.org so that it can be discussed?
msg380554 - (view) Author: Bryan Bishop (kanzure) Date: 2020-11-08 15:50
I'll go ahead and close this. The putrequest/putheader/endheaders suggestion is probably sufficient. Although I do wonder if a docs update is warranted, explaining the default behavior..
History
Date User Action Args
2020-11-08 15:50:40kanzuresetstatus: pending -> closed
resolution: wont fix
messages: + msg380554

stage: resolved
2020-11-07 02:38:45iritkatrielsetstatus: open -> pending
versions: + Python 3.9, Python 3.10, - Python 3.2, Python 3.3, Python 3.4
nosy: + iritkatriel

messages: + msg380489
2015-06-25 02:57:26martin.pantersetmessages: + msg245790
title: Add skip_host and skip_accept_encoding to httplib/http.client -> Add skip_host and skip_accept_encoding to HTTPConnection.request()
2014-09-10 07:23:46martin.pantersetnosy: + martin.panter
2013-01-01 18:40:10pitrousetmessages: + msg178743
2013-01-01 18:30:11kanzuresetmessages: + msg178741
2013-01-01 11:41:00pitrousetnosy: + orsenthil, pitrou
messages: + msg178730
2013-01-01 00:19:48berker.peksagsetversions: - Python 3.1, Python 2.7, Python 3.5
2012-12-31 23:57:53kanzurecreate