Message 359796 - Python tracker

➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author	larry
Recipients	Zooko.Wilcox-O'Hearn, christian.heimes, corona10, larry, xtreak
Date	2020-01-11.14:06:33
SpamBayes Score	-1.0
Marked as misclassified	Yes
Message-id	<1578751593.68.0.718003820723.issue39298@roundup.psfhosted.org>
In-reply-to

Content
For what it's worth, I spent some time producing clean benchmarks. All these were run on the same laptop, and all pre-load the same file (406668786 bytes) and run one update() on the whole thing to minimize overhead. K12 and BLAKE3 are using a hand-written C driver, and compiled with both gcc and clang; all the rest of the algorithms are from hashlib.new, python3 configured with --enable-optimizations and compiled with gcc. K12 and BLAKE3 support several SIMD extensions; this laptop only has AVX2 (no AVX512). All these numbers are the best of 3. All tests were run in a single thread. -----------------+----------+----------+----+----------------------- hash algorithm\|elapsed s \|mb/sec \|size\|hash -----------------+----------+----------+----+----------------------- K12-Haswell 0.176949 2298224495 64 24693954fa0dfb059f99... K12-Haswell-clang 0.181968 2234841926 64 24693954fa0dfb059f99... BLAKE3-AVX2-clang 0.250482 1623547723 64 30149a073eab69f76583... BLAKE3-AVX2 0.256845 1583326242 64 30149a073eab69f76583... md4 0.37684668 1079135924 32 d8a66422a4f0ae430317... sha1 0.46739069 870083193 40 a7488d7045591450ded9... K12-clang 0.498058 816509323 64 24693954fa0dfb059f99... BLAKE3 0.561470 724292378 64 30149a073eab69f76583... K12 0.569490 714093306 64 24693954fa0dfb059f99... BLAKE3-clang 0.573743 708800001 64 30149a073eab69f76583... blake2b 0.58276098 697831191 128 809ca44337af39792f8f... md5 0.59936016 678504863 32 306d7de4d1622384b976... sha384 0.64208886 633352818 96 b107ce5d086e9757efa7... sha512_224 0.66094102 615287556 56 90931762b9e553bd07f3... sha512_256 0.66465768 611846969 64 27b03aacdfbde1c2628e... sha512 0.6776549 600111921 128 f0af29e2019a6094365b... blake2s 0.86828375 468359318 64 02bee0661cd88aa2be15... sha256 0.97720436 416155312 64 48b5243cfcd90d84cd3f... sha224 1.0255457 396538907 56 10fb56b87724d59761c6... shake_128 1.0895037 373260576 32 2ec12727ac9d59c2e842... md5-sha1 1.1171806 364013470 72 306d7de4d1622384b976... sha3_224 1.2059123 337229156 56 93eaf083ca3a9b348e14... shake_256 1.3039152 311882857 64 b92538fd701791db8c1b... sha3_256 1.3417314 303092540 64 69354bf585f21c567f1e... ripemd160 1.4846368 273918025 40 30f2fe48fec404990264... sha3_384 1.7710776 229616579 96 61af0469534633003d3b... sm3 1.8384831 221198006 64 1075d29c75b06cb0af3e... sha3_512 2.4839673 163717444 128 c7c250e79844d8dc856e... If I can't have BLAKE3, I'm definitely switching to BLAKE2 ;-)

For what it's worth, I spent some time producing clean benchmarks.  All these were run on the same laptop, and all pre-load the same file (406668786 bytes) and run one update() on the whole thing to minimize overhead.  K12 and BLAKE3 are using a hand-written C driver, and compiled with both gcc and clang; all the rest of the algorithms are from hashlib.new, python3 configured with --enable-optimizations and compiled with gcc.  K12 and BLAKE3 support several SIMD extensions; this laptop only has AVX2 (no AVX512).  All these numbers are the best of 3.  All tests were run in a single thread.

-----------------+----------+----------+----+-----------------------
   hash algorithm|elapsed s |mb/sec    |size|hash
-----------------+----------+----------+----+-----------------------
      K12-Haswell 0.176949   2298224495  64  24693954fa0dfb059f99...
K12-Haswell-clang 0.181968   2234841926  64  24693954fa0dfb059f99...
BLAKE3-AVX2-clang 0.250482   1623547723  64  30149a073eab69f76583...
      BLAKE3-AVX2 0.256845   1583326242  64  30149a073eab69f76583...
              md4 0.37684668 1079135924  32  d8a66422a4f0ae430317...
             sha1 0.46739069  870083193  40  a7488d7045591450ded9...
        K12-clang 0.498058    816509323  64  24693954fa0dfb059f99...
           BLAKE3 0.561470    724292378  64  30149a073eab69f76583...
              K12 0.569490    714093306  64  24693954fa0dfb059f99...
     BLAKE3-clang 0.573743    708800001  64  30149a073eab69f76583...
          blake2b 0.58276098  697831191 128  809ca44337af39792f8f...
              md5 0.59936016  678504863  32  306d7de4d1622384b976...
           sha384 0.64208886  633352818  96  b107ce5d086e9757efa7...
       sha512_224 0.66094102  615287556  56  90931762b9e553bd07f3...
       sha512_256 0.66465768  611846969  64  27b03aacdfbde1c2628e...
           sha512 0.6776549   600111921 128  f0af29e2019a6094365b...
          blake2s 0.86828375  468359318  64  02bee0661cd88aa2be15...
           sha256 0.97720436  416155312  64  48b5243cfcd90d84cd3f...
           sha224 1.0255457   396538907  56  10fb56b87724d59761c6...
        shake_128 1.0895037   373260576  32  2ec12727ac9d59c2e842...
         md5-sha1 1.1171806   364013470  72  306d7de4d1622384b976...
         sha3_224 1.2059123   337229156  56  93eaf083ca3a9b348e14...
        shake_256 1.3039152   311882857  64  b92538fd701791db8c1b...
         sha3_256 1.3417314   303092540  64  69354bf585f21c567f1e...
        ripemd160 1.4846368   273918025  40  30f2fe48fec404990264...
         sha3_384 1.7710776   229616579  96  61af0469534633003d3b...
              sm3 1.8384831   221198006  64  1075d29c75b06cb0af3e...
         sha3_512 2.4839673   163717444 128  c7c250e79844d8dc856e...

If I can't have BLAKE3, I'm definitely switching to BLAKE2 ;-)

History
Date	User	Action	Args
2020-01-11 14:06:33	larry	set	recipients: + larry, christian.heimes, Zooko.Wilcox-O'Hearn, corona10, xtreak
2020-01-11 14:06:33	larry	set	messageid: <1578751593.68.0.718003820723.issue39298@roundup.psfhosted.org>
2020-01-11 14:06:33	larry	link	issue39298 messages
2020-01-11 14:06:33	larry	create