classification
Title: Fixing some byte-to-string conversion warnings
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: benjamin.peterson, eng793, ezio.melotti, pitrou, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2012-09-02 02:19 by eng793, last changed 2013-01-08 09:43 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
os_glob_bytes.patch eng793, 2012-09-02 02:19 patch review
Messages (7)
msg169683 - (view) Author: Alessandro Moura (eng793) * Date: 2012-09-02 02:19
This is related to issue 15826.

When run with the -b option, some glob.py and os.py functions give warnings due to byte-to-string conversions:


amoura@amoura-laptop:~/cpython$ ./python -b -c "import glob; glob.glob(b'cover*/glob.cover')"
/home/amoura/cpython/Lib/glob.py:64: BytesWarning: Comparison between bytes and string
  if basename == '':
amoura@amoura-laptop:~/cpython$ ./python -b -c "import os; os.makedirs(b'tst/making/dirs')"
/home/amoura/cpython/Lib/os.py:266: BytesWarning: Comparison between bytes and string
  if tail == cdir:           # xxx/newdir/. exists if xxx/newdir exists

The attached patch fixes this.

There is a rather more mysterious phenomenon with exceptions (which is triggered by test_exceptions for ImportException, but it happens for any Exception class):

>>> e = Exception(b'aaa')
[60596 refs]
>>> e.args[0]
b'aaa'
[60601 refs]
>>> str(e)
__main__:1: BytesWarning: str() on a bytes instance
"b'aaa'"
[60615 refs]
>>> e.args[0]
b'aaa'
[60615 refs]
>>> str(e)
"b'aaa'"
[60615 refs]
>>> e.args[0]
b'aaa'
[60615 refs]

In other words, if a bytes argument is given to the exception, the first call to str triggers the warning, but further calls don't. Is this worth pursuing?
msg169735 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-09-03 01:46
Not repeating warnings from the same place is the default warning behavior. You can get all of them by passing -Wall.
msg170868 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-09-21 09:06
-    if basename == '':
+    if len(basename) == 0:

This should be "if not basename:"


-        if tail == curdir:
+        cdir = curdir
+        if isinstance(tail, bytes):
+            cdir = bytes(curdir, 'ASCII')
+        if tail == cdir:

This will raise an error if curdir is a non-ascii str, so, unless the same error was already raised later in the code, this is backward incompatible.
msg177608 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-16 16:00
The first bug fixed in issue16696.
msg179278 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-07 19:46
> This will raise an error if curdir is a non-ascii str, so, unless the same error was already raised later in the code, this is backward incompatible.

On all supported platforms curdir is a ascii str (':' on Mac Classic, '.' on all other). The same idiom used in glob module.
msg179325 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-01-08 09:41
New changeset 9458a516f769 by Serhiy Storchaka in branch '3.2':
Issue #15845: Fix comparison between bytes and string.
http://hg.python.org/cpython/rev/9458a516f769

New changeset f6cf2985348a by Serhiy Storchaka in branch '3.3':
Issue #15845: Fix comparison between bytes and string.
http://hg.python.org/cpython/rev/f6cf2985348a

New changeset 51e60d9ee389 by Serhiy Storchaka in branch 'default':
Issue #15845: Fix comparison between bytes and string.
http://hg.python.org/cpython/rev/51e60d9ee389
msg179326 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-08 09:43
Fixed. Thank your for report and patch, Alessandro.
History
Date User Action Args
2013-01-08 09:43:37serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg179326

stage: patch review -> resolved
2013-01-08 09:41:11python-devsetnosy: + python-dev
messages: + msg179325
2013-01-07 19:46:29serhiy.storchakasetmessages: + msg179278
2013-01-07 19:09:18serhiy.storchakasetassignee: serhiy.storchaka
2012-12-16 16:00:16serhiy.storchakasetversions: + Python 3.2, Python 3.4
nosy: + serhiy.storchaka, pitrou

messages: + msg177608

type: enhancement -> behavior
2012-09-21 09:06:21ezio.melottisetmessages: + msg170868
2012-09-08 15:06:11ezio.melottisetnosy: + ezio.melotti

stage: patch review
2012-09-03 01:46:46benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg169735
2012-09-02 02:19:56eng793create