This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: SSL verification fails for some sites inside windows docker container
Type: behavior Stage: resolved
Components: SSL Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: SSL verification fails for some sites inside windows docker container
View: 36137
Assigned To: christian.heimes Nosy List: Mika Fischer, christian.heimes, matrixise, tianon
Priority: normal Keywords:

Created on 2019-02-27 14:34 by Mika Fischer, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (9)
msg336761 - (view) Author: Mika Fischer (Mika Fischer) Date: 2019-02-27 14:34
Inside a windows docker container, SSL verification fails for some but not all hosts. 

See this issue over in the docker repo:
https://github.com/docker-library/python/issues/359

Maybe you guys could shed some light on what could be the possible.

To reproduce, install Docker for Windows and then:

This works:
```
docker run -ti python:3.7-windowsservercore-1809 python -c "import urllib.request as r; r.urlopen('https://bootstrap.pypa.io').close()"
```

This doesn't
```
docker run -ti python:3.7-windowsservercore-1809 python -c "import urllib.request as r; r.urlopen('https://google.com').close()"
Traceback (most recent call last):
  File "C:\Python\lib\urllib\request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Python\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Python\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Python\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Python\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "C:\Python\lib\http\client.py", line 956, in send
    self.connect()
  File "C:\Python\lib\http\client.py", line 1392, in connect
    server_hostname=server_hostname)
  File "C:\Python\lib\ssl.py", line 412, in wrap_socket
    session=session
  File "C:\Python\lib\ssl.py", line 853, in _create
    self.do_handshake()
  File "C:\Python\lib\ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\Python\lib\urllib\request.py", line 543, in _open
    '_open', req)
  File "C:\Python\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Python\lib\urllib\request.py", line 1360, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "C:\Python\lib\urllib\request.py", line 1319, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>
```
msg336763 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2019-02-27 14:47
I can't reproduce it with python 3.8a2 and I don't have a laptop with Windows.
msg336764 - (view) Author: Mika Fischer (Mika Fischer) Date: 2019-02-27 14:51
Sorry if I wasn't completely clear. This issue occurs only on Windows and only when running python inside a windows docker container.

The question is what is python using on Windows to verify SSL certificates and what might be different inside the container that might let that process fail for some hosts and not for others...
msg336766 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2019-02-27 14:58
sure, I have just confirmed that this error does not occur with the last python 3.8a2 and that I don't have a Windows system for an eventual debugging session.
msg336767 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-02-27 15:05
This is probably a duplicate of #36137. Windows doesn't have CA certs pre-installed. They are downloaded from the update server on demand. Python doesn't trigger the update but only uses certs that are already present.

It's a design flaw in my implementation. I wasn't aware of Windows' behavior when I hooked up the Windows cert store to the SSL module.
msg336804 - (view) Author: Mika Fischer (Mika Fischer) Date: 2019-02-28 06:34
> This is probably a duplicate of #36137.

I think you mixed up the issue number, #36137 is this issue.

> Windows doesn't have CA certs pre-installed. They are downloaded from the update server on demand. Python doesn't trigger the update but only uses certs that are already present. It's a design flaw in my implementation. I wasn't aware of Windows' behavior when I hooked up the Windows cert store to the SSL module.

Oh, so that means that it is broken on normal Windows as well and only works coincidentally?

In any case, I can confirm that when the certificate is fetched using some other means, the request works:
```
docker run -ti python:3.7-windowsservercore-1809 powershell -command "(New-Object System.Net.WebClient).DownloadString(\"https://google.com\") >$null; python -c \"import urllib.request as r; r.urlopen('https://google.com').close()\""
```
msg336806 - (view) Author: Mika Fischer (Mika Fischer) Date: 2019-02-28 06:56
The easiest workaround I found (on Windows 10) is to use
```
certutil -generateSSTFromWU roots.sst && certutil -addstore -f root roots.sst && del roots.sst
```
before using Python...
msg336823 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-02-28 10:26
Err, I meant #36011

The enum cert store trick only breaks on a fresh installation. If you have used Windows for a bit (e.g. downloaded Firefox with Edge *g*), the root CA store is filled with common CA certs. Your certutil trick also works, as it triggers cert retrieval and update from Windows Update Server.
msg391395 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-19 19:55
I'm closing this issue as duplicate of #36137. There is no need to keep two issues open for the same problem.
History
Date User Action Args
2022-04-11 14:59:11adminsetgithub: 80318
2021-04-19 19:55:41christian.heimessetstatus: open -> closed
superseder: SSL verification fails for some sites inside windows docker container
messages: + msg391395

resolution: duplicate
stage: resolved
2021-04-19 19:55:41christian.heimeslinkissue36137 superseder
2019-02-28 10:26:28christian.heimessetmessages: + msg336823
2019-02-28 06:56:54Mika Fischersetmessages: + msg336806
2019-02-28 06:34:46Mika Fischersetmessages: + msg336804
2019-02-27 15:05:58christian.heimessetmessages: + msg336767
2019-02-27 15:00:15tianonsetnosy: + tianon
2019-02-27 14:58:53matrixisesetmessages: + msg336766
2019-02-27 14:51:49Mika Fischersetmessages: + msg336764
2019-02-27 14:47:50matrixisesetnosy: + matrixise
messages: + msg336763
2019-02-27 14:34:57Mika Fischercreate