This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients Joshua.J.Cogliati, brett.cannon, ezio.melotti, vstinner
Date 2014-05-13.01:28:24
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1399944506.53.0.531246293683.issue21401@psf.upfronthosting.co.za>
In-reply-to
Content
Attached py2_warn_cmp_bytes_text.patch adds BytesWarning for bytes == unicode, bytes != unicode, unicode == bytes, unicode != bytes and similar comparisons with bytearray. The new warnings are added when -b or -bb command line options are used.

As a consequence, a lot of tests are failing with the patch applied and -bb command line option.

Some tests are obviously wrong (unicode expected, but the tests use bytes), but it's much more complex to fix tricky modules like urllib, os.path, json and re (sre_parse) to handle "correctly" bytes and unicode. In some other cases, the warning should be made quiet because a same test compares bytes and then text.

It also means that programs currently working fine with Python 2.7.6 with "-3 -b" options will start to see new BytesWarning warnings. Is it acceptable?

What is the purpose of -b in Python 2? Help developers to notice earlier future Unicode issues in their program, or help them to port their code to Python 3?

Maybe the new warnings should only by emited if -3 and -b options are used at the same time?

Tell me if you would like to see my work-in-progress patch to fix the whole test suite. Just the stats:

$ hg diff --stat
 Lib/_pyio.py                         |  16 ++++++++--------
 Lib/ctypes/test/test_arrays.py       |  12 ++++++------
 Lib/ctypes/test/test_buffers.py      |  20 ++++++++++----------
 Lib/ctypes/test/test_cast.py         |   2 +-
 Lib/ctypes/test/test_memfunctions.py |  10 +++++-----
 Lib/ctypes/test/test_prototypes.py   |  10 +++++-----
 Lib/ctypes/test/test_structures.py   |   2 +-
 Lib/fractions.py                     |   1 +
 Lib/sqlite3/dump.py                  |   8 ++++----
 Lib/sqlite3/test/dump.py             |   4 ++--
 Lib/sre_parse.py                     |   3 ++-
 Lib/test/string_tests.py             |   6 +++---
 Lib/test/test_builtin.py             |  14 ++++++++++----
 Lib/test/test_bytes.py               |  27 +++++++++++++++++++++++++--
 Lib/test/test_format.py              |  10 +++++++---
 Lib/test/test_future4.py             |   2 +-
 Lib/test/test_pyexpat.py             |  28 ++++++++++++++--------------
 Lib/test/test_sax.py                 |  20 ++++++++++----------
 Lib/test/test_tempfile.py            |   4 ++--
 Objects/bytearrayobject.c            |   2 +-
 Objects/stringobject.c               |   9 +++++++++
 Objects/unicodeobject.c              |   8 ++++++++
 22 files changed, 135 insertions(+), 83 deletions(-)

A funny one:

diff -r 670fb496f1f6 Lib/test/test_future4.py
--- a/Lib/test/test_future4.py  Sun May 11 23:37:26 2014 -0400
+++ b/Lib/test/test_future4.py  Tue May 13 03:28:12 2014 +0200
@@ -43,5 +43,5 @@ class TestFuture(unittest.TestCase):
 def test_main():
     test_support.run_unittest(TestFuture)
 
-if __name__ == "__main__":
+if __name__ == b"__main__":
     test_main()
History
Date User Action Args
2014-05-13 01:28:26vstinnersetrecipients: + vstinner, brett.cannon, ezio.melotti, Joshua.J.Cogliati
2014-05-13 01:28:26vstinnersetmessageid: <1399944506.53.0.531246293683.issue21401@psf.upfronthosting.co.za>
2014-05-13 01:28:26vstinnerlinkissue21401 messages
2014-05-13 01:28:25vstinnercreate