Whereas openssl-based _hashlib refuses to accept unencoded strings:

>>> _hashlib.openssl_sha256("\xff")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object supporting the buffer API required

the _sha256 version encodes to UTF-8 by default:

>>> _sha256.sha256("\xff").digest() ==
_sha256.sha256("\xff".encode("utf-8")).digest()
True

I think refusing is better, but at least the behaviour should be
consistent. Same for the other algorithms in hashlib.

agreed. most platforms should be using the openssl version, i will
update the non-openssl implementations to behave the same.

I don't think this is worth being a release blocker. I'll do it for 3.0.1.

Seems that this problem is being taken care of in issue bpo-4751.

solved in bpo-4818 and bpo-4821

Fixed in py3k branch r69524.

needs porting to release30-maint.

possibly also release26-maint and trunk.

I don't think backporting to 2.6 is fine, people may be relying on the
current behaviour.
As for 3.0.1, you'd better be quick, it's scheduled for tomorrow.

Wooops, my mouse clicked on Remove!? I removed Message73550, sorry
gregory. Here was the content of the message:
---
agreed. most platforms should be using the openssl version, i will
update the non-openssl implementations to behave the same.

I don't think this is worth being a release blocker. I'll do it for
3.0.1.
---

I agree with pitrou: leave python 2.6 unchanged, but please backport
to 3.0.1 ;-)

gpolo gave me the solution to restore a deleted message:
http://bugs.python.org/issueXXXX?@action=edit&@add@messages=MSGNUM

fixed in release30-maint r69555.

sounds like its out of the question for 2.6. i will backport it to
trunk.

fixed in trunk r69561.

Gregory, this patch should not have been backported to Python 2.7. See issue

Could you please revert the change on trunk ? Thanks.

A much better solution would be to issue a -3 warning in case a Unicode
object is passed to the hash functions. However, this is major work to
get right, since the "s#" parser marker also accepts buffer interfaces.

What about inconsistent module build - as is reported some platform
build sha256 module that support unicode but most it is not build if
openssl is version 0.8+. Same for sha512 module.
If unicode for hashlib is not acceptable for trunk than why is not build
always sha{256|512} without to check for openssl version number ?

lemburg - see which issue #?

Anyways perhaps the right thing to do instead of trunk r65961 would have
been to change the s# to an s*.

Undoing it will be more painful now as several changes have gone in since
that require undoing and possibly redoing differently.

rpetrov - I couldn't really understand your message so I'm not sure if I'm
answering the right things: yes both the openssl and non-openssl modules
need to behave identically. the reason openssl is used when possible is
that its optimized hash functions are several times faster than the plain
C versions in the individual modules.

I think the missing issue reference is to this thread on python-dev:

http://mail.python.org/pipermail/python-dev/2009-December/094574.html

gregory - refer to setup.py logic to build modules

Gregory P. Smith wrote:

Gregory P. Smith <greg@krypto.org> added the comment:

lemburg - see which issue #?

Sorry, the message got truncated for some reason.

I was referring to http://bugs.python.org/issue3745

This was discussed on python-dev: http://mail.python.org/pipermail/python-dev/2009-December/094593.html

Anyways perhaps the right thing to do instead of trunk r65961 would have
been to change the s# to an s*.

That would have worked as well.

Undoing it will be more painful now as several changes have gone in since
that require undoing and possibly redoing differently.

Using s* should pretty much avoid the need to use GET_BUFFER_VIEW_OR_ERROUT(),
so if you want to keep the other changes, removing the use of the
macro should be fairly straight-forward, unless I'm missing something.

trunk r77252 switches python 2.7 to use 's*' for argument parsing. unicodes can be hashed (encoded to the system default encoding by s*) again.

This change has been blocked from being merged into py3k unless someone decides we actually want this magic unicode encoding behavior to exist there as well.

setup.py has also been updated to compile all versions of the hash algorithm modules when Py_DEBUG is defined. I'll update tests run on all implementations next so that it is easier for developers to maintain identical behavior across all implementations without needing to explicitly remember to reconfigure their setup and test those.

In order to get a -3 PyErr_WarnPy3k warning for unicode being passed to hashlib objects (a nice idea) I suggest creating an additonal 's*' like thing ('s3' perhaps?) in Python/getargs.c for that purpose rather than modifying all of the hashlib modules to accept an O, type check it and warn, and then re-parse it as a s* (that'd be a lot of tedious code duplication).

I believe everything in here has been addressed. Please open new issues with details for anything that doesn't quite right.

Gregory P. Smith wrote:

Gregory P. Smith <greg@krypto.org> added the comment:

trunk r77252 switches python 2.7 to use 's*' for argument parsing. unicodes can be hashed (encoded to the system default encoding by s*) again.

This change has been blocked from being merged into py3k unless someone decides we actually want this magic unicode encoding behavior to exist there as well.

Thanks for updating the implementation.

Gregory P. Smith wrote:

Gregory P. Smith <greg@krypto.org> added the comment:

In order to get a -3 PyErr_WarnPy3k warning for unicode being passed to hashlib objects (a nice idea) I suggest creating an additonal 's*' like thing ('s3' perhaps?) in Python/getargs.c for that purpose rather than modifying all of the hashlib modules to accept an O, type check it and warn, and then re-parse it as a s* (that'd be a lot of tedious code duplication).

Good idea. We're likely going to need this in more places, so I'm +1 on
adding an "s3" parser marker.