This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: urllib.request header characters being changed to lowercase
Type: behavior Stage: resolved
Components: Versions: Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: urllib2 forces title() on header names, breaking some requests
View: 12455
Assigned To: Nosy List: eric.smith, martin.panter, rosoroso
Priority: normal Keywords:

Created on 2019-11-17 14:38 by rosoroso, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (4)
msg356813 - (view) Author: (rosoroso) Date: 2019-11-17 14:38
When using the urllib request headers, I've found that certain letters are converted to their lowercase versions whilst part of a request object (H->h in the example, see at bottom).
Whilst this should not usually be an issue (Since headers are not supposed to be case sensitive), I've found an API for which that is not the case.

Options for fixing this could include, not changing the case of characters in header fields or updating the docs to include a warning of this behaviour (https://docs.python.org/3/library/urllib.request.html)

import urllib.request
header = {"Test-Header":"Value"}
requestobject = urllib.request.Request("https://www.example.com",None,header)
print ("Original header is:", header)
print ("Request header is:", requestobject.header_items())

'''
Orginal header is: {'Test-Header': 'Value'}
Request header is: [('Test-header', 'Value')]
Version was Python 3.6.5
'''
msg356823 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-11-17 18:04
Well, the standard says they're case insensitive: https://tools.ietf.org/html/rfc7230#section-3.2

Forcing the case on the header item seems quite deliberate: https://github.com/python/cpython/blob/master/Lib/urllib/request.py#L399
I assume that the .capitalize call is made in order to normalize headers, in case another header with the same name but different capitalization is added.

I can't recommend making a code change here. We're compliant with the standard, and any change would no doubt break someone's code.

I do agree that it might be worth documenting.

It's unfortunate that the caller can't specify an exact string they want to use (in order to handle broken servers), but that's the way it's been for ages.
msg356833 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2019-11-17 22:33
I suggest to keep the discussion with Issue 12455
msg356834 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-11-17 22:54
Yes, this does seem to be a duplicate. I'll close this.
History
Date User Action Args
2022-04-11 14:59:23adminsetgithub: 83012
2019-11-17 22:54:32eric.smithsetstatus: open -> closed
resolution: duplicate
messages: + msg356834

stage: resolved
2019-11-17 22:33:13martin.pantersetsuperseder: urllib2 forces title() on header names, breaking some requests

messages: + msg356833
nosy: + martin.panter
2019-11-17 18:04:25eric.smithsetnosy: + eric.smith
messages: + msg356823
2019-11-17 14:38:15rosorosocreate