classification
Title: hashlib fallback modules should be built even if openssl *is* available at build time
Type: enhancement Stage: resolved
Components: Build Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, dov, eric.araujo, fruch, gregory.p.smith, loewis, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2012-04-29 13:39 by dov, last changed 2012-05-16 14:45 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
hashlibfallbacks.patch pitrou, 2012-04-29 15:33 review
Messages (14)
msg159606 - (view) Author: Dov Feldstern (dov) Date: 2012-04-29 13:39
Python has its own implementations of various hash functions, that can be used as fallbacks when openssl is not available. However, if openssl *is* available at build time, then these fallbacks don't get built.

It would be nice if they would get built even if openssl *was* found at build time:

  (1) for (perceived?) licensing issues, one might choose to run without linking with openssl (see http://mail.python.org/pipermail/python-dev/2011-March/109053.html; that message requests that a bug be opened about this, but I wasn't able to find it?)

  (2) for "portable" builds: build onto portable or shared storage, and run on multiple machines, even if openssl happens not to be available on some of the machines.
msg159607 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-29 13:50
> that message requests that a bug be opened about this, but I wasn't able to find it?

Probably because no-one opened an issue.

> build onto portable or shared storage, and run on multiple machines,
> even if openssl happens not to be available on some of the machines.

For portable builds, I would suggest you ship OpenSSL as part of the build.
msg159617 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-29 15:33
Actually, the modules are always compiled when in debug mode. Here is a patch to always compile them.
msg159643 - (view) Author: Dov Feldstern (dov) Date: 2012-04-29 18:59
Thanks for the prompt response!

Yes, this looks good, I made a very similar patch myself and it seemed to work correctly (but that was on 2.7, and I didn't know how exactly to fix up the tests...).

Thanks again!
msg159646 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-04-29 20:21
The official way for building certain modules despite auto-configuration is to edit Modules/Setup, or Modules/Setup.local. This already supports the exact use case, so I'm closing this as "works for me".
msg159647 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-04-29 20:29
Antoine: why did you reopen it? Leave *at least* an explanation please.
msg159648 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-29 20:32
> Antoine: why did you reopen it? Leave *at least* an explanation please.

Ah, sorry. I think it's a reasonable enhancement request, and
furthermore the patch simplifies the setup code. 
Also, it allows us to test the _md5 (etc.) modules in non-debug mode as
well. Perhaps Gregory can tell us what he thinks.
msg159650 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-04-29 20:42
If it's an official feature that these modules are always built, the patch is insufficient: it then also needs to adjust the build environment for Windows, and the packaging. It potentially also affects packaging for OSX and Linux.

I still think that users with special needs such as (1) and (2) in the original message are better served by editig Modules/Setup, as that also allows e.g. to make the modules truly builtin.
msg159655 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2012-04-29 21:26
I don't have a problem with always compiling them.

Distro packagers should see that the stand alone versions are not distributed with their package that has a dependency on openssl as they'll just be a waste of space.  But that is up to the distro package maintainers.
msg159666 - (view) Author: Israel Fruchter (fruch) Date: 2012-04-30 04:28
I think (2) is very important, and I agree Gregory about the distro responsibility for size.

further more, if everything is define using the standard Modules/Setup, or Modules/Setup.local during compile/build time, why having a fallback anyhow in hashlib.py ?

BTW anyone that compiles for embedded systems adds a patch for remove all the code that detects modules automatically
see this from openembeded.org:

http://cgit.openembedded.org/openembedded/tree/recipes/python/python-2.6.6/nohostlibs.patch?id=9389f986ac8672cb671f00ab749ce323340b9500
msg159691 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2012-04-30 13:10
Python in Gentoo is also patched to always build _sha256, _sha512, _md5 and _sha1 modules.
msg159723 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2012-04-30 18:46
regarding the attached patch, rather than changing the test at all, I'd leave it as is.  The test as is will do what we want on the buildbots (warning us if they failed to compile when in debug mode).  I am not concerned about it testing if they compiled in opt mode or not.  I also do not think it is worth making them always compile on other platforms that do not use setup.py.

In short:  if we do anything, just make the setup.py change to get rid of the conditional compilation.
msg160872 - (view) Author: Roundup Robot (python-dev) Date: 2012-05-16 14:44
New changeset 0a1d7be10946 by Antoine Pitrou in branch 'default':
Issue #14693: Under non-Windows platforms, hashlib's fallback modules are always compiled, even if OpenSSL is present at build time.
http://hg.python.org/cpython/rev/0a1d7be10946
msg160873 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-05-16 14:45
Ok, should be fixed now. I fixed the proposed patch to avoid warnings when testing under Windows.
History
Date User Action Args
2012-05-16 14:45:13pitrousetstatus: open -> closed
resolution: fixed
messages: + msg160873

stage: patch review -> resolved
2012-05-16 14:44:15python-devsetnosy: + python-dev
messages: + msg160872
2012-05-04 16:54:10eric.araujosetnosy: + eric.araujo
2012-04-30 18:46:07gregory.p.smithsetmessages: + msg159723
2012-04-30 13:10:55Arfreversetnosy: + Arfrever
messages: + msg159691
2012-04-30 07:51:22loewissetversions: - Python 2.6, Python 2.7
2012-04-30 04:29:14fruchsetversions: + Python 2.6, Python 2.7
2012-04-30 04:28:50fruchsetnosy: + fruch
messages: + msg159666
2012-04-29 21:26:34gregory.p.smithsetmessages: + msg159655
2012-04-29 20:42:35loewissetmessages: + msg159650
2012-04-29 20:32:17pitrousetmessages: + msg159648
2012-04-29 20:29:39loewissetmessages: + msg159647
2012-04-29 20:22:59pitrousetstatus: closed -> open
resolution: works for me -> (no value)
stage: needs patch -> patch review
2012-04-29 20:21:42loewissetstatus: open -> closed

nosy: + loewis
messages: + msg159646

resolution: works for me
2012-04-29 18:59:09dovsetmessages: + msg159643
2012-04-29 15:33:32pitrousetfiles: + hashlibfallbacks.patch
keywords: + patch
messages: + msg159617
2012-04-29 13:55:32pitrousetstage: needs patch
2012-04-29 13:50:07pitrousetversions: + Python 3.3, - Python 2.7
nosy: + gregory.p.smith, pitrou

messages: + msg159607

type: enhancement
2012-04-29 13:39:32dovcreate