Title: install_certificates.command too complicated
Type: enhancement Stage: needs patch
Components: macOS Versions: Python 3.9, Python 3.8, Python 3.7, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: benjamin.peterson, dimpase, lukasz.langa, ned.deily, rhettinger, ronaldoussoren
Priority: critical Keywords:

Created on 2019-03-18 12:24 by dimpase, last changed 2019-08-26 09:52 by ned.deily.

Messages (7)
msg338211 - (view) Author: Dmitrii Pasechnik (dimpase) * Date: 2019-03-18 12:24
Currently (e.g. on the released Python 2.7.16) Mac/BuildScript/resources/install_certificates.command does install certifi module from the net and symlinks its cacert.pem to provide openssl with a working certificate. The same task may be accomplished much easier, by symlinking pip's cacert.pem, as follows (just shell commands, for the purposes of demonstration)

cd local/openssl
rm -f local/openssl/cert.pem
ln -s ../lib/python2.7/site-packages/pip/_vendor/certifi/cacert.pem cert.pem 

This works as pip's cacert.pem contains the same certificate as the one provided by unvendored certifi (as can be seen by looking at it using "openssl x509 -in ..." on it).

I'd be happy to provide a PR if this is acceptable.
msg338312 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-03-19 02:49
Thanks for the suggestion but that is not a workable solution for two reasons. One, pip is an optional install with the installer so we cannot depend on it being available.  More importantly, from a packaging point of view, the internal composition of pip is opaque.  There's no guarantee that any future release of pip will still bundle root certificates, that they will be installed in the same location, or which root certificates will be included and how up-to-date they are. Pip uses the certificates primarily to access PyPI, not to provide a general set of root certificates.  The current solution of providing the Install Certificates script as an example is certainly far from ideal and we will improve it.  But adding a dependency on undocumented behavior of pip is not a step in the right direction.
msg338325 - (view) Author: Dmitrii Pasechnik (dimpase) * Date: 2019-03-19 07:53
The script install_certificates.command depends upon pip, it calls pip to install certifi. Thus it's no less "optional" than pip.

And pip is only functional, and it able to do the installation in question,  due to it including the certificate in question.

The role of this script is fishy from security point of view, too. 
Why not simply putting the certificate right where it belongs to, i.e. not just simplify install_certificates.command, but simply get rid of it?
msg338752 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-03-24 20:43
I do not disagree that the current manual Install Certificates step is not ideal but, again, for the reasons cited in my earlier response (and other reasons), adding a dependency on pip to provide certificates is not a good idea.  But, since there does not seem to be another open issue about this right now, I am going to reopen this one and use it to implement a solution that eliminates the need to manually run Install Certificates at installation time.
msg338771 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-03-24 23:53
> I am going to reopen this one and use it to implement a solution 
> that eliminates the need to manually run Install Certificates
> at installation time.

There will be much rejoicing.  Almost every week, I have a learner bump into this issue.
msg350296 - (view) Author: Ɓukasz Langa (lukasz.langa) * (Python committer) Date: 2019-08-23 14:12
This is marked as a release blocker. The last 3.8 beta is scheduled for Monday. Please decide how to proceed ASAP.
msg350516 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-08-26 09:52
So as not to delay 3.8.0b4, I'm removing this as a "release blocker'. Once the implementation is in master, we can consider backporting to other releases.
Date User Action Args
2019-08-26 09:52:18ned.deilysetpriority: release blocker -> critical

messages: + msg350516
versions: + Python 3.9
2019-08-23 14:12:38lukasz.langasetpriority: deferred blocker -> release blocker
nosy: + lukasz.langa, benjamin.peterson
messages: + msg350296

2019-03-24 23:53:34rhettingersetnosy: + rhettinger
messages: + msg338771
2019-03-24 20:46:00ned.deilylinkissue36340 superseder
2019-03-24 20:43:55ned.deilysetstatus: closed -> open
priority: normal -> deferred blocker

assignee: ned.deily
title: install_certificates.command too complicated, copy from pip's dir instead -> install_certificates.command too complicated
resolution: rejected ->
versions: + Python 3.7, Python 3.8
messages: + msg338752
stage: resolved -> needs patch
2019-03-19 07:53:31dimpasesetmessages: + msg338325
2019-03-19 02:49:26ned.deilysetstatus: open -> closed
resolution: rejected
messages: + msg338312

stage: resolved
2019-03-18 12:24:20dimpasecreate