classification
Title: py3k-pep3137: issue warnings / errors on str(bytes()) and similar operations
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.0
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gvanrossum Nosy List: christian.heimes, gvanrossum
Priority: high Keywords: patch

Created on 2007-11-05 14:22 by christian.heimes, last changed 2008-01-06 22:29 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
py3k-pep3137_remove_str_bytes.patch christian.heimes, 2007-11-05 14:22
py3k-pep3137_remove_str_bytes2.patch christian.heimes, 2007-11-05 15:17
py3k-pep3137_str_bytes_ascii.patch christian.heimes, 2007-11-05 16:27
py3k-pep3137_str_bytes_warning.patch christian.heimes, 2007-11-05 19:46
py3k-pep3137_str_bytes_warning2.patch christian.heimes, 2007-11-06 02:16
Messages (16)
msg57124 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-05 14:22
str(bytes()) == repr(bytes()) and str(buffer()) == repr(buffer()) is
causing a bunch bugs which are extremely hard to understand. On several
occasions I didn't understand the problem until I removed a str() call
or made str(bytes()) and str(buffer()) raise an exception.
msg57125 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-05 14:31
Err, please remove the (reprfunc) cast and check the language of the
error messages, too :)
msg57126 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-05 15:17
Here is an updated patch that fixes an error in httplib that was causing
the xmlrpc test suite to block.
msg57127 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-11-05 15:51
I'll look at the patches later, but we've gone over this before on the
list. str() of *any* object needs to return *something*. Yes, it's
unfortunate that this masks bugs in the transitional period, but it
really is the best thing in the long run. We had other exceptional
treatement for str vs. bytes (e.g. the comparison was raising TypeError
for a while) and we had to kill that too.
msg57128 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-05 15:59
Guido van Rossum wrote:
> I'll look at the patches later, but we've gone over this before on the
> list. str() of *any* object needs to return *something*. Yes, it's
> unfortunate that this masks bugs in the transitional period, but it
> really is the best thing in the long run. We had other exceptional
> treatement for str vs. bytes (e.g. the comparison was raising TypeError
> for a while) and we had to kill that too.

Can we agree to a compromise and make str(bytes()) return
bytes().decode("ascii")? I think it's a sensible default behavior and
catches the errors in the code I've seen so far.

Christian
msg57129 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-05 16:27
New patch:

static PyObject *
string_str(PyObject *op)
{
	return PyObject_CallMethod(op, "decode", "s", "ascii");
}
msg57131 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-11-05 17:39
No -- making str(b) return b.decode("ascii") brings back the same issues
we had with mindlessly mixing PyString and PyUnicode in 2.x.  That was a
major disaster.
msg57132 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-11-05 18:10
A compromise I could live with: add a command line option that makes it
a warning.
msg57133 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-05 19:25
Guido van Rossum wrote:
> Guido van Rossum added the comment:
> 
> A compromise I could live with: add a command line option that makes it
> a warning.

Good idea. I can live with it, too. :) How do you like

-b: issue warnings about str(bytes_instance) and str(buffer_instance)
    (-bb: issue errors)

Christian
msg57134 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-05 19:46
I've made a patch with -b -> warning and -bb > type error. I'm not happy
with the warning message. Maybe you've a better idea for it.
msg57136 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-11-05 20:44
The error message is fine, though you could drop the word "Calling"
without loss of information. ;-) Also, most warnings don't seem to use
a trailing period.

Perhaps you could expand this to also add a trap in the comparison
function when Py{String,Bytes} is compared to PyUnicode.
msg57137 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-05 21:00
Guido van Rossum wrote:
> The error message is fine, though you could drop the word "Calling"
> without loss of information. ;-) Also, most warnings don't seem to use
> a trailing period.
> 
> Perhaps you could expand this to also add a trap in the comparison
> function when Py{String,Bytes} is compared to PyUnicode.

It's sounds like a reasonable idea. But first I'm going to hit the down
with my gf and meet some friends. Cu later!

PS: In my humble opinion Amaury Forgeot d'Arc deserves an entry in the
Misc/ACKS list for his patches.
msg57138 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-11-05 21:04
> It's sounds like a reasonable idea. But first I'm going to hit the down
> with my gf and meet some friends. Cu later!

No! You're not allowed to have a life! :-)

(And no hacking drunk, either. :-)

> PS: In my humble opinion Amaury Forgeot d'Arc deserves an entry in the
> Misc/ACKS list for his patches.

Agreed -- done!
msg57153 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-06 02:16
Pfff! I've written my best code after having one or two beers :)

Updates:
* Added class BytesWarning(Warning)
* Added -b/-bb command line args and Py_BytesWarningFlag
* issue PyErr_WarnEx(PyExc_BytesWarning) when Py_BytesWarningFlag is set.
* Added warning filter for BytesWarning that raises an exception if
Py_BytesWarningFlag > 1

Open questions:
* the warning filter isn't set when the initializer cannot load the
warning module (e.g. frozen apps). W/o the warning module Python is
screwed anyway and frozen apps would never add the -b argument.
* the warnings are only enabled with -b. They can't be enabled by
modifying the warnings.filters. Is this fine with you?
msg57154 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-11-06 03:44
> Updates:
> * Added class BytesWarning(Warning)
> * Added -b/-bb command line args and Py_BytesWarningFlag
> * issue PyErr_WarnEx(PyExc_BytesWarning) when Py_BytesWarningFlag is set.
> * Added warning filter for BytesWarning that raises an exception if
> Py_BytesWarningFlag > 1
>
> Open questions:
> * the warning filter isn't set when the initializer cannot load the
> warning module (e.g. frozen apps). W/o the warning module Python is
> screwed anyway and frozen apps would never add the -b argument.
> * the warnings are only enabled with -b. They can't be enabled by
> modifying the warnings.filters. Is this fine with you?

Yes, this is is all fine with me. Feel free to submit overnight.
msg57160 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-06 11:53
Applied to py3k-pep3137 in r 58876
History
Date User Action Args
2008-01-06 22:29:45adminsetkeywords: - py3k
versions: Python 3.0
2007-11-06 11:53:53christian.heimessetstatus: closed
resolution: fixed
messages: + msg57160
title: py3k-pep3137: str(bytes()) and str(buffer()) should raise TypeError patch -> py3k-pep3137: issue warnings / errors on str(bytes()) and similar operations
2007-11-06 03:44:21gvanrossumsetmessages: + msg57154
2007-11-06 02:16:12christian.heimessetfiles: + py3k-pep3137_str_bytes_warning2.patch
messages: + msg57153
2007-11-05 21:04:00gvanrossumsetmessages: + msg57138
2007-11-05 21:00:30christian.heimessetmessages: + msg57137
2007-11-05 20:44:23gvanrossumsetmessages: + msg57136
2007-11-05 19:46:51christian.heimessetfiles: + py3k-pep3137_str_bytes_warning.patch
messages: + msg57134
2007-11-05 19:25:48christian.heimessetmessages: + msg57133
2007-11-05 18:10:14gvanrossumsetmessages: + msg57132
2007-11-05 17:39:08gvanrossumsetstatus: open -> (no value)
messages: + msg57131
2007-11-05 16:27:04christian.heimessetfiles: + py3k-pep3137_str_bytes_ascii.patch
messages: + msg57129
2007-11-05 15:59:55christian.heimessetmessages: + msg57128
2007-11-05 15:51:31gvanrossumsetmessages: + msg57127
2007-11-05 15:17:25christian.heimessetfiles: + py3k-pep3137_remove_str_bytes2.patch
messages: + msg57126
2007-11-05 14:31:08christian.heimessetmessages: + msg57125
2007-11-05 14:22:51christian.heimescreate