This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python 3 logging HTTPHandler sends duplicate Host header
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: lhelwerd, redstone-cold, vinay.sajip
Priority: normal Keywords: patch

Created on 2017-07-11 14:11 by lhelwerd, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4465 merged vinay.sajip, 2017-11-19 18:02
PR 4468 merged python-dev, 2017-11-19 18:42
Messages (4)
msg298161 - (view) Author: Leon Helwerda (lhelwerd) Date: 2017-07-11 14:11
The logging HTTPHandler sends two Host headers which confuses certain servers.

Tested versions:
Python 3.6.1
lighttpd/1.4.45

Steps to reproduce (MWE):

1) Set up a lighttpd server which is to act as the logging host (we do not actually implement anything that accepts the input here). Optionally enable the debug settings from https://redmine.lighttpd.net/projects/1/wiki/DebugVariables (specifically, add debug.log-condition-handling = "enable" to /etc/lighttpd/lighttpd.conf) to follow what is happening inside the server.
2) In python3:
import logging
import logging.handlers
handler = logging.handlers.HTTPHandler('localhost', '/')
logging.getLogger().setLevel(logging.INFO)
logging.getLogger().addHandler(handler)
logging.info('hello world')
3) Notice that the access logs in /var/log/lighttpd/access.log show a 400 response for the request (in Python, the response is ignored). If the debugging from 1) is enabled, then /var/log/lighttpd/error.log contains a line "duplicate Host-header -> 400".

This is not a bug in lighttpd. The server adheres to RFC7320 (sec. 5.4, p. 44): "A server MUST respond with a 400 (Bad Request) status code [...] to any request message that contains more than one Host header field".

A workaround is to put the full URL in the second argument of HTTPRequest:
handler = logging.handlers.HTTPHandler('localhost', 'http://localhost/')
Then lighttpd follows RFC2616 (sec. 5.2, p. 37): "If Request-URI is an absoluteURI, the host is part of the Request-URI. Any Host header field value in the request MUST be ignored."

The origin of this issue is that the http.client.HTTPConnection.putrequest method (called by HTTPHandler.emit) already adds a Host header unless skip_host=True is given. Thus the manual addition of a Host header is duplicate.

Other versions like Python 2.7 might also be affected but I did not test.
msg306507 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2017-11-19 18:36
New changeset e96ba183c43ad6633b5d014b3dc57433e2802faf by Vinay Sajip in branch 'master':
bpo-30904: Removed duplicated Host: header. (#4465)
https://github.com/python/cpython/commit/e96ba183c43ad6633b5d014b3dc57433e2802faf
msg306511 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2017-11-19 19:16
New changeset b071a5e838a0e84c4e8a60448fbd40e8a7e5c882 by Vinay Sajip (Miss Islington (bot)) in branch '3.6':
bpo-30904: Removed duplicated Host: header. (GH-4465) (#4468)
https://github.com/python/cpython/commit/b071a5e838a0e84c4e8a60448fbd40e8a7e5c882
msg307835 - (view) Author: iMath (redstone-cold) Date: 2017-12-08 04:41
Yes, I met with the same bug, see the post for description and bug investigation 
https://stackoverflow.com/questions/43185804/using-httphandler-cause-django-server-side-shows-http-400-and-invalid-http-host/47434323#47434323
History
Date User Action Args
2022-04-11 14:58:48adminsetgithub: 75087
2017-12-08 04:41:14redstone-coldsetnosy: + redstone-cold
messages: + msg307835
2017-11-19 22:27:39vinay.sajipsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-11-19 19:16:19vinay.sajipsetmessages: + msg306511
2017-11-19 18:42:24python-devsetpull_requests: + pull_request4400
2017-11-19 18:36:19vinay.sajipsetmessages: + msg306507
2017-11-19 18:02:39vinay.sajipsetkeywords: + patch
stage: patch review
pull_requests: + pull_request4398
2017-10-26 17:09:27vinay.sajipsetversions: + Python 3.7
2017-10-26 07:32:10berker.peksagsetnosy: + vinay.sajip
2017-07-11 14:11:00lhelwerdcreate