Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(38759)

Delta Between Two Patch Sets: Doc/library/hashlib.rst

Issue 16113: Add SHA-3 (Keccak) support
Left Patch Set: Created 6 years, 11 months ago
Right Patch Set: Created 3 years, 1 month ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
Left: Side by side diff | Download
Right: Side by side diff | Download
« no previous file with change/comment | « no previous file | Lib/hashlib.py » ('j') | no next file with change/comment »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
LEFTRIGHT
1 :mod:`hashlib` --- Secure hashes and message digests 1 :mod:`hashlib` --- Secure hashes and message digests
2 ==================================================== 2 ====================================================
3 3
4 .. module:: hashlib 4 .. module:: hashlib
5 :synopsis: Secure hash and message digest algorithms. 5 :synopsis: Secure hash and message digest algorithms.
6
6 .. moduleauthor:: Gregory P. Smith <greg@krypto.org> 7 .. moduleauthor:: Gregory P. Smith <greg@krypto.org>
7 .. sectionauthor:: Gregory P. Smith <greg@krypto.org> 8 .. sectionauthor:: Gregory P. Smith <greg@krypto.org>
8 9
10 **Source code:** :source:`Lib/hashlib.py`
9 11
10 .. index:: 12 .. index::
11 single: message digest, MD5 13 single: message digest, MD5
12 single: secure hash algorithm, SHA1, SHA224, SHA256, SHA384, SHA512 14 single: secure hash algorithm, SHA1, SHA224, SHA256, SHA384, SHA512
13 15
14 **Source code:** :source:`Lib/hashlib.py` 16 .. testsetup::
17
18 import hashlib
19
15 20
16 -------------- 21 --------------
17 22
18 This module implements a common interface to many different secure hash and 23 This module implements a common interface to many different secure hash and
19 message digest algorithms. Included are the FIPS secure hash algorithms SHA1, 24 message digest algorithms. Included are the FIPS secure hash algorithms SHA1,
20 SHA224, SHA256, SHA384, and SHA512 (defined in FIPS 180-2) as well as RSA's MD5 25 SHA224, SHA256, SHA384, and SHA512 (defined in FIPS 180-2) as well as RSA's MD5
21 algorithm (defined in Internet :rfc:`1321`). The terms "secure hash" and 26 algorithm (defined in Internet :rfc:`1321`). The terms "secure hash" and
22 "message digest" are interchangeable. Older algorithms were called message 27 "message digest" are interchangeable. Older algorithms were called message
23 digests. The modern term is secure hash. 28 digests. The modern term is secure hash.
24 29
25 .. note:: 30 .. note::
26 If you want the adler32 or crc32 hash functions they are available in 31
32 If you want the adler32 or crc32 hash functions, they are available in
27 the :mod:`zlib` module. 33 the :mod:`zlib` module.
28 34
29 .. warning:: 35 .. warning::
30 36
31 Some algorithms have known hash collision weaknesses, see the FAQ at the end. 37 Some algorithms have known hash collision weaknesses, refer to the "See
38 also" section at the end.
39
40
41 .. _hash-algorithms:
42
43 Hash algorithms
44 ---------------
32 45
33 There is one constructor method named for each type of :dfn:`hash`. All return 46 There is one constructor method named for each type of :dfn:`hash`. All return
34 a hash object with the same simple interface. For example: use :func:`sha1` to 47 a hash object with the same simple interface. For example: use :func:`sha256` to
35 create a SHA1 hash object. You can now feed this object with objects conforming 48 create a SHA-256 hash object. You can now feed this object with :term:`bytes-lik e
36 to the buffer interface (normally :class:`bytes` objects) using the 49 objects <bytes-like object>` (normally :class:`bytes`) using the :meth:`update` method.
37 :meth:`update` method. At any point you can ask it for the :dfn:`digest` of the 50 At any point you can ask it for the :dfn:`digest` of the
38 concatenation of the data fed to it so far using the :meth:`digest` or 51 concatenation of the data fed to it so far using the :meth:`digest` or
39 :meth:`hexdigest` methods. 52 :meth:`hexdigest` methods.
40 53
41 .. note:: 54 .. note::
42 55
43 For better multithreading performance, the Python GIL is released for 56 For better multithreading performance, the Python :term:`GIL` is released for
44 strings of more than 2047 bytes at object creation or on update. 57 data larger than 2047 bytes at object creation or on update.
45 58
46 .. note:: 59 .. note::
47 60
48 Feeding string objects is to :meth:`update` is not supported, as hashes work 61 Feeding string objects into :meth:`update` is not supported, as hashes work
49 on bytes, not on characters. 62 on bytes, not on characters.
50 63
51 .. index:: single: OpenSSL; (use in module hashlib) 64 .. index:: single: OpenSSL; (use in module hashlib)
52 65
53 Constructors for hash algorithms that are always present in this module are 66 Constructors for hash algorithms that are always present in this module are
54 :func:`md5`, :func:`sha1`, :func:`sha224`, :func:`sha256`, :func:`sha384`, 67 :func:`sha1`, :func:`sha224`, :func:`sha256`, :func:`sha384`,
55 :func:`sha512`, :func:`sha3_224`, :func:`sha3_256`, :func:`sha3_384`, and 68 and :func:`sha512`. :func:`md5` is normally available as well, though it
56 :func:`sha3_512`. Additional algorithms may also be available depending upon 69 may be missing if you are using a rare "FIPS compliant" build of Python.
57 the OpenSSL library that Python uses on your platform. 70 Additional algorithms may also be available depending upon the OpenSSL
58 71 library that Python uses on your platform. On most platforms the
59 .. versionchanged:: 3.4 72 :func:`sha3_224`, :func:`sha3_256`, :func:`sha3_384`, :func:`sha3_512`,
60 Add sha3 family of hash algorithms. 73 :func:`shake_128`, :func:`shake_256` are also available.
74
75 .. versionadded:: 3.6
76 SHA3 (Keccak) and SHAKE constructors :func:`sha3_224`, :func:`sha3_256`,
77 :func:`sha3_384`, :func:`sha3_512`, :func:`shake_128`, :func:`shake_256`.
61 78
62 For example, to obtain the digest of the byte string ``b'Nobody inspects the 79 For example, to obtain the digest of the byte string ``b'Nobody inspects the
63 spammish repetition'``:: 80 spammish repetition'``::
64 81
65 >>> import hashlib 82 >>> import hashlib
66 >>> m = hashlib.md5() 83 >>> m = hashlib.sha256()
67 >>> m.update(b"Nobody inspects") 84 >>> m.update(b"Nobody inspects")
68 >>> m.update(b" the spammish repetition") 85 >>> m.update(b" the spammish repetition")
69 >>> m.digest() 86 >>> m.digest()
70 b'\xbbd\x9c\x83\xdd\x1e\xa5\xc9\xd9\xde\xc9\xa1\x8d\xf0\xff\xe9' 87 b'\x03\x1e\xdd}Ae\x15\x93\xc5\xfe\\\x00o\xa5u+7\xfd\xdf\xf7\xbcN\x84:\xa6\xaf \x0c\x95\x0fK\x94\x06'
71 >>> m.digest_size 88 >>> m.digest_size
72 16 89 32
73 >>> m.block_size 90 >>> m.block_size
74 64 91 64
75 92
76 More condensed: 93 More condensed:
77 94
78 >>> hashlib.sha224(b"Nobody inspects the spammish repetition").hexdigest() 95 >>> hashlib.sha224(b"Nobody inspects the spammish repetition").hexdigest()
79 'a4337bc45a8fc544c03f52dc550cd6e1e87021bc896588bd79e901e2' 96 'a4337bc45a8fc544c03f52dc550cd6e1e87021bc896588bd79e901e2'
80 97
81 .. function:: new(name[, data]) 98 .. function:: new(name[, data])
82 99
83 Is a generic constructor that takes the string name of the desired 100 Is a generic constructor that takes the string name of the desired
84 algorithm as its first parameter. It also exists to allow access to the 101 algorithm as its first parameter. It also exists to allow access to the
85 above listed hashes as well as any other algorithms that your OpenSSL 102 above listed hashes as well as any other algorithms that your OpenSSL
86 library may offer. The named constructors are much faster than :func:`new` 103 library may offer. The named constructors are much faster than :func:`new`
87 and should be preferred. 104 and should be preferred.
88 105
89 Using :func:`new` with an algorithm provided by OpenSSL: 106 Using :func:`new` with an algorithm provided by OpenSSL:
90 107
91 >>> h = hashlib.new('ripemd160') 108 >>> h = hashlib.new('ripemd160')
92 >>> h.update(b"Nobody inspects the spammish repetition") 109 >>> h.update(b"Nobody inspects the spammish repetition")
93 >>> h.hexdigest() 110 >>> h.hexdigest()
94 'cc4a5ce1b3df48aec5d22d1f16b894a0b894eccc' 111 'cc4a5ce1b3df48aec5d22d1f16b894a0b894eccc'
95 112
96 Hashlib provides the following constant attributes: 113 Hashlib provides the following constant attributes:
97 114
98 .. data:: algorithms_guaranteed 115 .. data:: algorithms_guaranteed
99 116
100 Contains the names of the hash algorithms guaranteed to be supported 117 A set containing the names of the hash algorithms guaranteed to be supported
101 by this module on all platforms. 118 by this module on all platforms. Note that 'md5' is in this list despite
119 some upstream vendors offering an odd "FIPS compliant" Python build that
120 excludes it.
102 121
103 .. versionadded:: 3.2 122 .. versionadded:: 3.2
104 123
105 .. data:: algorithms_available 124 .. data:: algorithms_available
106 125
107 Contains the names of the hash algorithms that are available 126 A set containing the names of the hash algorithms that are available in the
108 in the running Python interpreter. These names will be recognized 127 running Python interpreter. These names will be recognized when passed to
109 when passed to :func:`new`. :attr:`algorithms_guaranteed` 128 :func:`new`. :attr:`algorithms_guaranteed` will always be a subset. The
110 will always be a subset. Duplicate algorithms with different 129 same algorithm may appear multiple times in this set under different names
111 name formats may appear in this set (thanks to OpenSSL). 130 (thanks to OpenSSL).
112 131
113 .. versionadded:: 3.2 132 .. versionadded:: 3.2
114 133
115 The following values are provided as constant attributes of the hash objects 134 The following values are provided as constant attributes of the hash objects
116 returned by the constructors: 135 returned by the constructors:
117 136
118 137
119 .. data:: hash.digest_size 138 .. data:: hash.digest_size
120 139
121 The size of the resulting hash in bytes. 140 The size of the resulting hash in bytes.
122 141
123 .. data:: hash.block_size 142 .. data:: hash.block_size
124 143
125 The internal block size of the hash algorithm in bytes. 144 The internal block size of the hash algorithm in bytes.
145
146 A hash object has the following attributes:
147
148 .. attribute:: hash.name
149
150 The canonical name of this hash, always lowercase and always suitable as a
151 parameter to :func:`new` to create another hash of this type.
152
153 .. versionchanged:: 3.4
154 The name attribute has been present in CPython since its inception, but
155 until Python 3.4 was not formally specified, so may not exist on some
156 platforms.
126 157
127 A hash object has the following methods: 158 A hash object has the following methods:
128 159
129 160
130 .. method:: hash.update(arg) 161 .. method:: hash.update(arg)
131 162
132 Update the hash object with the object *arg*, which must be interpretable as 163 Update the hash object with the object *arg*, which must be interpretable as
133 a buffer of bytes. Repeated calls are equivalent to a single call with the 164 a buffer of bytes. Repeated calls are equivalent to a single call with the
134 concatenation of all the arguments: ``m.update(a); m.update(b)`` is 165 concatenation of all the arguments: ``m.update(a); m.update(b)`` is
135 equivalent to ``m.update(a+b)``. 166 equivalent to ``m.update(a+b)``.
136 167
137 .. versionchanged:: 3.1 168 .. versionchanged:: 3.1
138 The Python GIL is released to allow other threads to run while hash 169 The Python GIL is released to allow other threads to run while hash
139 updates on data larger than 2048 bytes is taking place when using hash 170 updates on data larger than 2047 bytes is taking place when using hash
140 algorithms supplied by OpenSSL. 171 algorithms supplied by OpenSSL.
141 172
142 173
143 .. method:: hash.digest() 174 .. method:: hash.digest()
144 175
145 Return the digest of the data passed to the :meth:`update` method so far. 176 Return the digest of the data passed to the :meth:`update` method so far.
146 This is a bytes object of size :attr:`digest_size` which may contain bytes in 177 This is a bytes object of size :attr:`digest_size` which may contain bytes in
147 the whole range from 0 to 255. 178 the whole range from 0 to 255.
148 179
149 180
150 .. method:: hash.hexdigest() 181 .. method:: hash.hexdigest()
151 182
152 Like :meth:`digest` except the digest is returned as a string object of 183 Like :meth:`digest` except the digest is returned as a string object of
153 double length, containing only hexadecimal digits. This may be used to 184 double length, containing only hexadecimal digits. This may be used to
154 exchange the value safely in email or other non-binary environments. 185 exchange the value safely in email or other non-binary environments.
155 186
156 187
157 .. method:: hash.copy() 188 .. method:: hash.copy()
158 189
159 Return a copy ("clone") of the hash object. This can be used to efficiently 190 Return a copy ("clone") of the hash object. This can be used to efficiently
160 compute the digests of data sharing a common initial substring. 191 compute the digests of data sharing a common initial substring.
161 192
162 193
194 SHAKE variable length digests
195 -----------------------------
196
197 The :func:`shake_128` and :func:`shake_256` algorithms provide variable
198 length digests with length_in_bits//2 up to 128 or 256 bits of security.
199 As such, their digest methods require a length. Maximum length is not limited
200 by the SHAKE algorithm.
201
202 .. method:: shake.digest(length)
203
204 Return the digest of the data passed to the :meth:`update` method so far.
205 This is a bytes object of size ``length`` which may contain bytes in
206 the whole range from 0 to 255.
207
208
209 .. method:: shake.hexdigest(length)
210
211 Like :meth:`digest` except the digest is returned as a string object of
212 double length, containing only hexadecimal digits. This may be used to
213 exchange the value safely in email or other non-binary environments.
214
215
216 Key derivation
217 --------------
218
219 Key derivation and key stretching algorithms are designed for secure password
220 hashing. Naive algorithms such as ``sha1(password)`` are not resistant against
221 brute-force attacks. A good password hashing function must be tunable, slow, and
222 include a `salt <https://en.wikipedia.org/wiki/Salt_%28cryptography%29>`_.
223
224
225 .. function:: pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None)
226
227 The function provides PKCS#5 password-based key derivation function 2. It
228 uses HMAC as pseudorandom function.
229
230 The string *hash_name* is the desired name of the hash digest algorithm for
231 HMAC, e.g. 'sha1' or 'sha256'. *password* and *salt* are interpreted as
232 buffers of bytes. Applications and libraries should limit *password* to
233 a sensible length (e.g. 1024). *salt* should be about 16 or more bytes from
234 a proper source, e.g. :func:`os.urandom`.
235
236 The number of *iterations* should be chosen based on the hash algorithm and
237 computing power. As of 2013, at least 100,000 iterations of SHA-256 are
238 suggested.
239
240 *dklen* is the length of the derived key. If *dklen* is ``None`` then the
241 digest size of the hash algorithm *hash_name* is used, e.g. 64 for SHA-512.
242
243 >>> import hashlib, binascii
244 >>> dk = hashlib.pbkdf2_hmac('sha256', b'password', b'salt', 100000)
245 >>> binascii.hexlify(dk)
246 b'0394a2ede332c9a13eb82e9b24631604c31df978b4e2f0fbd2c549944f9d79a5'
247
248 .. versionadded:: 3.4
249
250 .. note::
251
252 A fast implementation of *pbkdf2_hmac* is available with OpenSSL. The
253 Python implementation uses an inline version of :mod:`hmac`. It is about
254 three times slower and doesn't release the GIL.
255
256
163 .. seealso:: 257 .. seealso::
164 258
165 Module :mod:`hmac` 259 Module :mod:`hmac`
166 A module to generate message authentication codes using hashes. 260 A module to generate message authentication codes using hashes.
167 261
168 Module :mod:`base64` 262 Module :mod:`base64`
169 Another way to encode binary hashes for non-binary environments. 263 Another way to encode binary hashes for non-binary environments.
170 264
171 http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf 265 http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf
172 The FIPS 180-2 publication on Secure Hash Algorithms. 266 The FIPS 180-2 publication on Secure Hash Algorithms.
173 267
174 http://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_a lgorithms 268 https://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_ algorithms
175 Wikipedia article with information on which algorithms have known issues a nd 269 Wikipedia article with information on which algorithms have known issues a nd
176 what that means regarding their use. 270 what that means regarding their use.
177 271
272 https://www.ietf.org/rfc/rfc2898.txt
273 PKCS #5: Password-Based Cryptography Specification Version 2.0
LEFTRIGHT
« no previous file | Lib/hashlib.py » ('j') | Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Toggle Comments ('s')

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+