As far as I can tell _ssl works properly. However, test_ssl returns FAIL at some very basic levels, e.g.
...
test_constructor (test.test_ssl.ContextTests) ... ERROR
...
test_protocol (test.test_ssl.ContextTests) ... ERROR
test_python_ciphers (test.test_ssl.ContextTests) ... ok
test_session_stats (test.test_ssl.ContextTests) ... ERROR

When using applications that depend on python (e.g., git) and getting "SSL" related errors - doing 

export SSL_CERT_FILE=/var/ssl/somefile.pem

the problems go away. However, it looks asif that variable is not being used by python (3.7).

Given: AIX openssl does not have a default CAFile nor CAPath, etc., only that openssl.cnf is at /var/ssl/openssl.cnf. Also - AIX openssl.base does not include any certificates.

Question: does python have a documented (or undocumented) env variable it uses to look for, provide, or override a system/distribution default?

Any comments re: environment variables - even if the answer is None!

update: went back to check what worked, did not work without the environment variable set.

I am going to guess that pip(3) is able to make use of the environment variable SSL_CERT_FILE as pip download fails (in some cases) without it, but succeeds with it.

I thought to recall something similar while using git (mine leaning on python2-2.7) but I have not had the time to test it again (using git fetch and git pull from cpython).

I "guess" it is somewhere in this code. But I am getting lost in all the macros that call other macros.

Some help would really be appreciated!

Currently looking in _ssl.c at:

/*[clinic input]
_ssl.get_default_verify_paths

Return search paths and environment vars that are used by SSLContext's set_default_verify_paths() to load defa
ult CAs.

The values are 'cert_file_env', 'cert_file', 'cert_dir_env', 'cert_dir'.
[clinic start generated code]*/

static PyObject *
_ssl_get_default_verify_paths_impl(PyObject *module)
/*[clinic end generated code: output=e5b62a466271928b input=5210c953d98c3eb5]*/
{
    PyObject *ofile_env = NULL;
    PyObject *ofile = NULL;
    PyObject *odir_env = NULL;
    PyObject *odir = NULL;

#define CONVERT(info, target) { \
        const char *tmp = (info); \
        target = NULL; \
        if (!tmp) { Py_INCREF(Py_None); target = Py_None; } \
        else if ((target = PyUnicode_DecodeFSDefault(tmp)) == NULL) { \
            target = PyBytes_FromString(tmp); } \
        if (!target) goto error; \
    }

    CONVERT(X509_get_default_cert_file_env(), ofile_env);
    CONVERT(X509_get_default_cert_file(), ofile);
    CONVERT(X509_get_default_cert_dir_env(), odir_env);
    CONVERT(X509_get_default_cert_dir(), odir);
#undef CONVERT

    return Py_BuildValue("NNNN", ofile_env, ofile, odir_env, odir);

  error:
    Py_XDECREF(ofile_env);
    Py_XDECREF(ofile);
    Py_XDECREF(odir_env);
    Py_XDECREF(odir);
    return NULL;
}

What I would like to know is what environment variable is being used. Not clear to me from the code here.

Thx.

On 04/08/2018 16:37, Michael Felt wrote:
> Some help would really be appreciated!

Gotten a bit further :)

While it does not affect the 'failures', this change decreases 'errors'
by 8 (skipped +1).

I do not expect this to be 'acceptable' - however, I hope this helps an
expert come with some advice.

I played around with defining either OPENSSL_NO_SSL2 or
OPENSSL_VERSION_1_1. However, I do not think the latter is correct (AIX
still goes it - externally, openssl.1.0.2.XXXX, not openssl.1.1.Y.XXXX)
and I felt
the configure process was attempting to use a dynamic process to
establish OPENSSL_NO_SSL2
rather than a definition being added to CFLAGS.

Again - help appreciated!

Before:
FAILED (failures=13, errors=11, skipped=10)
test test_ssl failed

After:
FAILED (failures=13, errors=2, skipped=11)
test test_ssl failed

diff --git a/Modules/_ssl.c b/Modules/_ssl.c
index 2bce4816d2..5fa442cedf 100644
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -5790,9 +5790,11 @@ PyInit__ssl(void)

     /* protocol versions */
 #ifndef OPENSSL_NO_SSL2
+#ifndef _AIX
     PyModule_AddIntConstant(m, "PROTOCOL_SSLv2",
                             PY_SSL_VERSION_SSL2);
 #endif
+#endif
 #ifndef OPENSSL_NO_SSL3
     PyModule_AddIntConstant(m, "PROTOCOL_SSLv3",
                             PY_SSL_VERSION_SSL3);

On 21/08/2018 09:46, Michael wrote:
> On 04/08/2018 16:37, Michael Felt wrote:
>> Some help would really be appreciated!
> Gotten a bit further :)

A little bit more:

Modules/_ssl.c

 +3707  fprintf(stderr,"load_cert_chain():certfile:%s\n", (char *)
PyBytes_AS_STRING(certfile_bytes));
 +3708      PySSL_BEGIN_ALLOW_THREADS_S(pw_info.thread_state);
 +3709      r = SSL_CTX_use_certificate_chain_file(self->ctx,
 +3710          PyBytes_AS_STRING(certfile_bytes));
 +3711      PySSL_END_ALLOW_THREADS_S(pw_info.thread_state);
 +3712      if (r != 1) {
 +3713  fprintf(stderr,"load_cert_chain():r:%d: errno:%d
ERR_peek_last_error():%d\n", r, errno, ERR_peek_last_error());

load_cert_chain():certfile:/data/prj/python/git/python3-3.8/Lib/test/XXXnonexisting.pem
load_cert_chain():r:0: errno:2 ERR_peek_last_error():0
load_cert_chain():certfile:/data/prj/python/git/python3-3.8/Lib/test/nullcert.pem
load_cert_chain():r:0: errno:0 ERR_peek_last_error():0

Note: I swapped BADCERT and NULLCERT, so now above shows with NULLCERT,
while below

* Below: the first failure - is an OSError (file does not exist, and
passes the test). The second test is "badcert" and AIX is not reporting
the error via ERR_peek_last_error(), but is does seem there is an error
that 'openssl' does return. The third is just to show a connection where
CAfile provides the needed data (for comparison)

FIRST: works as expected

root@x066:[/data/prj/python/python3-3.8]openssl s_client -quiet -connect
www.mindrot.org:443 -CAfile
/data/prj/python/git/python3-3.8/Lib/test/XXXnonex>
804401144:error:02001002:system library:fopen:No such file or
directory:bss_file.c:175:fopen('/data/prj/python/git/python3-3.8/Lib/test/XXXnonexisting.pem','r')
804401144:error:2006D080:BIO routines:BIO_new_file:no such
file:bss_file.c:182:
804401144:error:0B084002:x509 certificate
routines:X509_load_cert_crl_file:system lib:by_file.c:253:
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify error:num=20:unable to get local issuer certificate

SECOND: there are errors, but not one reported by ERR_peek_last_error()?

BADCERT
root@x066:[/data/prj/python/python3-3.8]openssl s_client -quiet -connect
www.mindrot.org:443 -CAfile
/data/prj/python/git/python3-3.8/Lib/test/badcert.>
804401144:error:0906D064:PEM routines:PEM_read_bio:bad base64
decode:pem_lib.c:830:
804401144:error:0B084009:x509 certificate
routines:X509_load_cert_crl_file:PEM lib:by_file.c:259:
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify error:num=20:unable to get local issuer certificate

NULLCERT
root@x066:[/data/prj/python/python3-3.8]openssl s_client -quiet -connect
www.mindrot.org:443 -CAfile
/data/prj/python/git/python3-3.8/Lib/test/nullcert>
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify error:num=20:unable to get local issuer certificate

THIRD: working as expected, for comparison

root@x066:[/data/prj/python/python3-3.8]openssl s_client -quiet -connect
www.mindrot.org:443 -CAfile /var/ssl/cacert.pem
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = mindrot.org
verify return:1

Again - help requested!!!

Michael

When built against a less optimized OpenSSL library all tests pass. So, IMHO, not a bug, and closing.

The buildbots will (eventually) build against a less optimized library and the error messages will match. That was the cause of all these messages (no matching error message).