classification
Title: Patches for thread-support in built-in SHA modules
Type: performance Stage: patch review
Components: Extension Modules Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, ebfe, gregory.p.smith, jcea, vstinner
Priority: normal Keywords: patch

Created on 2009-01-03 17:08 by ebfe, last changed 2016-06-12 11:22 by christian.heimes.

Files
File name Uploaded Description Edit
sha1module_small_locks.diff ebfe, 2009-01-03 17:08
sha256module_small_locks.diff ebfe, 2009-01-03 17:08
sha512module_small_locks.diff ebfe, 2009-01-03 17:09
Messages (6)
msg78975 - (view) Author: Lukas Lueg (ebfe) Date: 2009-01-03 17:08
Here is the follow-up to issue #4818. The patches attached allow the
built-in SHA modules to release the GIL.

Also the build-in SHA modules will now no longer accept "s#" as input.
Input is parsed just as in the openssl-driven classes where
unicode-objects are explicitly rejected.


The built-in hash modules have been not quite beautiful before even more
code is now copy & pasted between them. Is there any interest in
refactoring all those modules? AFAIK _sha1 and such are only used by
hashlib.py ...
msg79146 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-01-05 13:58
This patch is a duplicate of #3745.
msg79148 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-01-05 14:00
Oops, it's not a duplicate /o\ But it may solves #3745 (reject unicode 
in sha256).
msg79995 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-01-17 02:20
sha1module_small_locks.diff patch is very similar to the changes made 
in #4751, except:
 - SHA1_GIL_MINSIZE is 8192 whereas HASHLIB_GIL_MINSIZE is 2048
 - There is no test for PyThread_allocate_lock() failure

Instead of copy/paste code in hashlib, sha1, sha256 and sha512 (4 
modules), can't we share some constants, functions or macros? 
Examples:
 - the GIL minimum size constant
 - the long MY_GET_BUFFER_VIEW_OR_ERROUT macro (which can be a 
function)

And about sha, why using 3 files for sha? Are the source code so 
different? In the GNU libc, they use "template" files (it's possible 
even with the C language using the preprocessor!): strtof(), strtod() 
and strtold() share 99% of the source code. Interesting content of 
strtof.c :
----
#define	FLOAT		float
#define	FLT		FLT
#ifdef USE_WIDE_CHAR
#define STRTOF		wcstof
#define STRTOF_L	__wcstof_l
#else
# define STRTOF		strtof
# define STRTOF_L	__strtof_l
#endif

#include "strtod.c"
----

Refactoring to share code between hash modules will ease the changes, 
eg. release the GIL ;-)
msg81728 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2009-02-12 07:40
fyi - I took care of the unicode data acceptance issue for all hashlib 
related modules in the py3k branch in r69524.

Yes, the hashlib modules have a -lot- of code in common.  Refactoring 
that would be nice but i haven't looked into it.  I at least moved the 
commonly used #define into a new hashlib.h for the above change.
msg172069 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2012-10-05 10:28
I'll integrate your patch once I'm done with my SHA-3 patch #16113. I'm using parts of your patch in my new sha3 code to release the GIL.

I'll also check if I can share more code between the SHA family modules.
History
Date User Action Args
2016-06-12 11:22:32christian.heimessetassignee: christian.heimes ->
2016-04-22 18:58:35christian.heimessetversions: + Python 3.6, - Python 3.4
2012-10-05 10:28:44christian.heimessetassignee: christian.heimes
type: performance
components: + Extension Modules

nosy: + christian.heimes
messages: + msg172069
stage: patch review
2012-10-04 21:25:35christian.heimessetversions: + Python 3.4
2009-02-12 07:40:34gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg81728
2009-01-29 11:27:44jceasetnosy: + jcea
2009-01-17 02:20:46vstinnersetmessages: + msg79995
2009-01-05 14:00:39vstinnersetmessages: + msg79148
2009-01-05 13:58:56vstinnersetnosy: + vstinner
messages: + msg79146
2009-01-03 17:09:06ebfesetfiles: + sha512module_small_locks.diff
2009-01-03 17:08:52ebfesetfiles: + sha256module_small_locks.diff
2009-01-03 17:08:41ebfesetfiles: + sha1module_small_locks.diff
keywords: + patch
2009-01-03 17:08:23ebfecreate