Sometimes I am using httplib/http.client and the server is not exactly conforming to HTTP specs. I need to be able to specify the exact headers that are sent to the server. By default, httplib/http.client injects headers like "Host" and "Accept-Encoding".

Issue #831747 added skip_accept_encoding to httplib's putrequest method, but not on the request method. This current patch exposes these two toggles on the request method. This way, headers can be controlled without manually calling the connect/send/endheaders methods.

As a result, urllib/urllib3 can call urlopen and pass in these attributes to more directly control the headers sent to the remote server.

I'm not sure this is a desirable feature in the stdlib, but regardless, your solution isn't very scalable: a new argument will have to be added each time someone wants to avoid sending a given header.

Another possibility would be to allow passing None in values of the `headers` dict, in which case the given header wouldn't be send at all.

On Tue, Jan 1, 2013 at 5:41 AM, Antoine Pitrou wrote:
> Another possibility would be to allow passing None in values of the `headers` dict, in which case the given header wouldn't be send at all.

I agree that your solution is more scaling-friendly than the patch I originally submitted. But from another perspective, consider how alien it is to have to explicitly add keys set to None in the headers to get desired behavior.

Maybe there could be a single flag "strict" (or possibly a better name) that would indicate that the dict is in fact what I want to send and that it should not be modified? Or perhaps there should be "base_headers" list (not a dictionary) of header keys of which defaults to not inject?

Also, would making {"Accept-Encoding": None} send no "Accept-Encoding" header be backwards incompatible with the current behavior?

> On Tue, Jan 1, 2013 at 5:41 AM, Antoine Pitrou wrote:
> > Another possibility would be to allow passing None in values of the
> `headers` dict, in which case the given header wouldn't be send at
> all.
> 
> I agree that your solution is more scaling-friendly than the patch I
> originally submitted. But from another perspective, consider how alien
> it is to have to explicitly add keys set to None in the headers to get
> desired behavior.

Well, that is not a common need, is it?
That said, we could indeed have a separate argument "omit_headers"
listing the headers that we don't want to send.

I tend to think that this proposal should be rejected. If you need low-level control over the header fields, just use putrequest(), putheader() and endheaders() methods instead. And adding these flags to request() isn’t going to help with urlopen(), unless the Request class was extended in a similar way.

Bryan, as there isn't unanimous agreement about the feature you suggested: if you are still interested in working on this, could you bring it up on python-ideas@python.org so that it can be discussed?

I'll go ahead and close this. The putrequest/putheader/endheaders suggestion is probably sufficient. Although I do wonder if a docs update is warranted, explaining the default behavior..