New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python 3.7.0 wont compile with SSL Support 1.1.0 > alledged missing X509_VERIFY_PARAM_set1_host() support #78209
Comments
when compiling Python 3.7.0 setup.py is reporting that the ssl module failed to compile due to missing support for X509_VERIFY_PARAM_set1_host() despite it existing in rsa.h for all versions of OpenSSL 1.1.0. Could not build the ssl module! In addition _ssl.o does actually compile. The issue appears that _ssl is appearing in "missing", "self.failed", "self.failed_on_import" setup.py
I havent had time to go through the code yet to find out where the error is gettng flagged and if its a associated with how I have compiled openssl i.e. I need a compilation flag to enabled X509_VERIFY_PARAM_set1_host() support. |
The function definition should be in openssl/x509_vfy.h, not rsa.h. What's the output of configure on your system? You should see something like ./configure What's your platform and openssl version? |
Apologies, my bad you are correct the function was defined in x509_vfy.h Im compiling on RHEL I havce tried checking for pkg-config... /usr/bin/pkg-config My details of Setup are; SSL=/home/{my_home_folder}/openssl I now spot that the Linker is having issues (-L) libssl.so & cyypt.so is in /home/{my_home_folder}/openssl =/home/{my_home_folder}/openssl/include/openssl including opensslconf.h DEPRECATEDIN_1_0_0 etc etc |
configure is not able to find OpenSSL. You either have to configure Python to pick up your OpenSSL (./configure --with-openssl=/path/to/openssl) or install the OpenSSL developer packages. RHEL 7.5 comes with OpenSSL 1.0.2, so you are good. |
Thanks I have found teh root cause of the problem ... --with-openssl=[my_dir] The configure scripts has an assumption you are compiling against a binary packaged version of openssl and that there is a /lib folder under [my_dir]. This simply does not exist under any of the source code releases of openssl. So after I compiled the openssl source code I had to create the lib folder under my openssh build directory and symlink the *.so libraries there for the configure script to work This is still an issue even if you edit Setup correctlty to compile the module.
To make the code more robust should it not 1st check under the root of [my_dir] before assuming [my_dir]/lib exests or at least report teh full path with the /lib added onto teh end of {my_dir} so you know where confiure has gone wrong ? Is this not a fair expectation? no lib folder with lib folder Thanks for all your help |
autoconf's --with-library options typically don't support build directories and work with installed versions only. The --with-openssl is no different. I suggest that you install OpenSSL to a local directory and then configure Python to fetch OpenSSL from that directory. The multissltest script in Tools/ssl uses that approach to build Python with multiple OpenSSL versions. |
The configure script doesn't work with a proper openssl installation either. Even though there is a "lib" directory in the directory given to --with-openssl=<myssl>, libssl.so.1.1 isn't found, because there is still a "-L<myssl>/lib" missing in some of the compiler calls. LDFLAGS="-L<myssl>/lib" ./configure --with-openssl=<myssl> is required, which seems somehow redundant. Bug? |
I was unable to get it working even with all the suggestions in this thread. I have a shared account on a system with only Python 2.7 and an old version of openssl. I have write access only to my user directory. I installed a new openssl in a local directory and pointed to it with both --with-openssl and LDFLAGS, as suggested. The configure step seems to work, but on make the libssl.so.1.1 still isn't found. I fell back to Python 3.6. Same result. I fell back to 3.4. It finally worked. |
I had to add $HOME/usr/lib64 to LD_LIBRARY_PATH to get make to work. |
OS: RHEL 6.8 I installed OpenSSL 1.1.1b from source into /usr/local. Because it's RHEL, the libs are in /usr/local/lib64 (as set up by default with the OpenSSL "make install") which the configure script does not seem to know about. My workaround: before running configure for Python, set the environment variable: LDFLAGS="-L/usr/local/lib -L/usr/local/lib64 -Wl,-rpath,/usr/local/lib -Wl,-rpath,/usr/local/lib64" Once that is set, configure manages to find the proper libssl: checking for openssl/ssl.h in /usr/local... yes |
I have the same issue installing v3.7.3 on RHEL6.8. The standard version came with openssl v1.0.1c, which would not configure. I installed openssl 1.0.2s in /usr/local and created a file /etc/profile.d/openssl.sh adding the following lines: # /etc/profile.d/openssl.sh Exiting and relogging into the shell, the version returns openssl 1.0.2s 28 May 2019. As has been mentioned, there is no openssl distro out there that contains the filepath that either configure or make appears to expect. To get through configuration, I began with configure, modifying the ssldirs variable to /usr/local/openssl, and also repointing the following to the appropriate subdirs: 17214 if ! $found; then This apparently wasn't sufficient for configure to recognize this openssl installation. Next, I uncommented and modified Modules/Setup.dist to reflect the openssl header and lib paths: 211 SSL=/usr/local/openssl That gets us to here with ./configure: checking whether compiling and linking against OpenSSL works... yes However using ./configure -with-openssl=/usr/local/openssl, configure returns: checking whether compiling and linking against OpenSSL works... yes So at least from configure's standpoint, I was able to get configure to pick up the openssl folder and its include/openssl which contains x509_vfy.h. Ok great. However make doesn't appear to respect these changes in configure - I get one of the typical variants of: *** WARNING: renaming "_hashlib" since importing it failed: libssl.so.1.0.0: cannot open shared object file: No such file or directory Python build finished successfully! The following modules found by detect_modules() in setup.py, have been Failed to build these modules: Following modules built successfully but were removed because they could not be imported: Could not build the ssl module! What I'd like to know is, what is the difference between what configure is looking for and in what cases would make not necessarily respect the confirmations of configure's checks? Let me know if there are any dumps/logs you'd be interested in. I realize that in my case we're dealing with an older OS, but this issue doesn't seem restricted per se to that alone. |
I'm having a similar problem. I'm trying to compile on Red Hat 6.9 using a locally installed OpenSSL library. I've tried everything here (I think) and I still get this error: checking for openssl/ssl.h in /home/mf/dp/mpma/ghantousm/apptron/local/... yes I've ensured the libssl.* files are all present or linked to in /home/mf/dp/mpma/ghantousm/apptron/local/lib and .../lib64, the include files are present, the OpenSSL build seemed fine, and I compiled version 1.1.1d of openssl. I've tried setting LDFLAGS and rpath and still no dice; the setup.py module doesn't seem to have any hard-coded paths so I'm really at a loss to understand what is wrong. I've tried both versions 3.8.0 and 3.7.3 of python with the --with-openssl=/home/mf/dp/mpma/ghantousm/apptron/local option set, and version 3.6.8 (which doesn't accept any ssl related options). This bug seems to have been preserved for a number of versions, so I'd like to help squash it if I can, but not sure where to go to from here. I notice that some of the reports here suggest that they've had success with work-arounds, but as I stated none of them seem to work for me and I'm unable to compile with ssl support. |
Python uses https://www.gnu.org/software/autoconf-archive/ax_check_openssl.html to detect and check for OpenSSL. Please check config.log for any errors. The log file will contain an error message. How did you compile OpenSSL? Did you configure the sources with "./config shared" ? config.log: make: ./python
>>> import _ssl
>>> _ssl._OPENSSL_API_VERSION
(1, 1, 0, 12, 15) |
Have you also tried $ yum install openssl-devel ? That should work without requiring to compile openssl from source, unless you want a later version, which isn't advisable to install system-wide, as it could break other things. |
Thanks for your responses. Yum is not an option as I'm on a machine without root privileges. OK, so I have to be a bit contrite here, I tried everything you suggested, but in the end the solution goes back to this comment, and the LDFLAGS environment variable: I think I was using the wrong compiler when I tried that solution the first time. So I'm sorry about that. But while we're here, I may as well ask if there isn't another way to do this, without having recourse to the environment variable. Is there a way to specify these flags using just the configure script? Based on something I read somewhere on the web, I generally prefer to steer clear of setting environment variables, though perhaps I needn't be so reluctant. Incidentally, the environment variables OPENSSL_LDFLAGS etc as suggested in config.log (thanks for that, I wouldn't have thought to look there) do not appear to work. |
--with-openssl only sets the header location for the pre-processor and library path for the dynamic linker. It does *not* affect the search and lookup paths of the dynamic loader! If you have installed OpenSSL in a non-standard location then you need to tell the dynamic loader how to load the shared libraries. This can be done in three ways:
|
hello, As per "Python 3.7.0 will not compile on RHEL6 because it requires OpenSSL 1.0.2 or 1.1 and RHEL6 provides 1.0.1e" openssl version confirms this to be the case on Ubuntu 14.04 "On Ubuntu 14.04 on Dreamhost, an extra flag is required for Python 3.7+: then I ran: after that python3.7 was correct hope that helps, kind regards |
I couldn't get this to work at all, python 3.7 compiled fine, but at the end it reports: ''' Python build finished successfully! Following modules built successfully but were removed because they could not be imported: Could not build the ssl module! But in the end I got it to work like this: ----- install_python3.7.sh ---- mkdir /tmp/openssl echo /usr/local/openssl1.1.1/lib > /etc/ld.so.conf.d/openssl1.1.1.conf mkdir /tmp/python37 ldconfig This important pieces are: echo /usr/local/openssl1.1.1/lib > /etc/ld.so.conf.d/openssl1.1.1.conf to make it find the .so to load it at runtime and ./configure --with-ensurepip=yes --with-openssl=/usr/local/openssl1.1.1 CFLAGS="-I/usr/local/openssl1.1.1/include" LDFLAGS="-L/usr/local/openssl1.1.1/lib" CXX=/usr/bin/g++ specifying the non-standard openssl-version specifically. |
That's a very dangerous trick and I advise against it. You are modifying the global linker path and inject custom OpenSSL libraries into it. This may affect and disrupt other programs or OS core tools. Instead compile the _ssl and _hashlib module with rpath, e.g. LD_RUN_PATH. You also don't have to modify CFLAGS or LDFLAGS. --with-openssl does that for you. $ export LD_RUN_PATH=/home/heimes/dev/python/multissl/openssl/1.1.1f/lib
$ ./configure --with-openssl=/home/heimes/dev/python/multissl/openssl/1.1.1f -C
$ make
$ unset LD_RUN_PATH
$ ldd build/lib.linux-x86_64-3.9/_ssl.cpython-39-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffc124eb000)
libssl.so.1.1 => /home/heimes/dev/python/multissl/openssl/1.1.1f/lib/libssl.so.1.1 (0x00007fd3d7cab000)
libcrypto.so.1.1 => /home/heimes/dev/python/multissl/openssl/1.1.1f/lib/libcrypto.so.1.1 (0x00007fd3d7974000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fd3d791c000)
libc.so.6 => /lib64/libc.so.6 (0x00007fd3d7753000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fd3d774c000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd3d7d8e000) |
Python 3.10 contains various improvements that make it easier to compile and link Python with a custom OpenSSL installation. You can find more information in ticket bpo-43466. |
good! |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: