classification
Title: -b option undocumented
Type: Stage: resolved
Components: Documentation Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Arfrever, docs@python, emily.zhao, ezio.melotti, gbengeult, georg.brandl, lemburg, martin.panter, merwok, ncoghlan, r.david.murray, zach.ware
Priority: normal Keywords: easy, patch

Created on 2011-03-26 00:53 by merwok, last changed 2017-05-13 14:46 by zach.ware. This issue is now closed.

Files
File name Uploaded Description Edit
issue11681.patch emily.zhao, 2014-06-07 21:59 review
b_option.patch gbengeult, 2017-01-11 21:17 adds codumentation for -b and -bb to main.c and cmdline.rst review
Pull Requests
URL Status Linked Edit
PR 1562 merged zach.ware, 2017-05-12 21:01
Messages (19)
msg132195 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2011-03-26 00:53
Python 2 has a -b command-line option which is undocumented.

It is used by the warnings machinery to decide to warn about (-b) or raise an error (-bb) if someone uses str(bytes) or str(buffer) (explanation courtesy of grep and Python/pythonrun.c).
msg132197 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-03-26 01:05
Are you sure?  (see issue 10471).  I don't see how -b could do anything in python2, since bytes and string are the same there (and, indeed, from command line experimentation it doesn't seem to).
msg132198 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2011-03-26 01:08
The code definitely is in 2.7.

Python/_warnings.c:861:    if (Py_BytesWarningFlag > 1)
Python/_warnings.c:863:    else if (Py_BytesWarningFlag)
Python/_warnings.c:867:    PyList_SET_ITEM(filters, pos++, create_filter(PyExc_BytesWarning,
Python/sysmodule.c:1257:    SetFlag(Py_BytesWarningFlag);
Python/pythonrun.c:81:int Py_BytesWarningFlag; /* Warn on str(bytes) and str(buffer) */
Include/pydebug.h:14:PyAPI_DATA(int) Py_BytesWarningFlag;
Include/pyerrors.h:178:PyAPI_DATA(PyObject *) PyExc_BytesWarning;
Modules/main.c:285:            Py_BytesWarningFlag++;
Objects/bytearrayobject.c:1003:    if (Py_BytesWarningFlag) {
Objects/bytearrayobject.c:1004:        if (PyErr_WarnEx(PyExc_BytesWarning,
Objects/bytearrayobject.c:1028:        if (Py_BytesWarningFlag && op == Py_EQ) {
Objects/bytearrayobject.c:1029:            if (PyErr_WarnEx(PyExc_BytesWarning,
Objects/exceptions.c:2018: *    BytesWarning extends Warning
Objects/exceptions.c:2020:SimpleExtendsException(PyExc_Warning, BytesWarning,
Objects/exceptions.c:2110:    PRE_INIT(BytesWarning)
Objects/exceptions.c:2178:    POST_INIT(BytesWarning)
Lib/warnings.py:399:    simplefilter(bytes_action, category=BytesWarning, append=1)
Lib/test/test_bytes.py:25:            with test.test_support.check_warnings(('', BytesWarning)):
Lib/test/exception_hierarchy.txt:50:	   +-- BytesWarning
Include/pydebug.h:14:PyAPI_DATA(int) Py_BytesWarningFlag;
Include/pyerrors.h:178:PyAPI_DATA(PyObject *) PyExc_BytesWarning;
Doc/library/warnings.rst:164:* :exc:`BytesWarning` is ignored unless the :option:`-b` option is given once or
msg132200 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-03-26 01:12
rdmurray@hey:~/python/p27>cat temp.py
x = bytes('abc')
print x
print str(x)
rdmurray@hey:~/python/p27>./python -bb temp.py
abc
abc
msg132201 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2011-03-26 01:17
test_bytes has a branch for -b and passes without it, with -b and with -bb.  I really don’t know.
msg132300 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-03-27 09:57
Even "from __future__ import unicode_literals" doesn't make it do anything.

Perhaps Christian merged it by mistake in [5341b30b1812]?
msg132354 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2011-03-27 18:34
Nick Coghlan wrote:
> 
> Nick Coghlan <ncoghlan@gmail.com> added the comment:
> 
> Even "from __future__ import unicode_literals" doesn't make it do anything.
> 
> Perhaps Christian merged it by mistake in [5341b30b1812]?

It looks more like some parts were left out in the merge
by accident: The 2.7 code only raises the warning for bytearrays,
not for bytes and buffers, whereas 3.2 issues the warning for
all bytes and bytearray (but not buffers; which makes sense,
since Python 3 no longer has a text buffer interface).
msg164089 - (view) Author: moijes12 (moijes12) Date: 2012-06-26 18:26
Hi Marc

I tried reproducing this for bytearray using Python 2.7.2 but I can't see a warning.

devel@moses:~$ python --version
Python 2.7.2+
devel@moses:~$ cat test_py.py 
k = range(10)
kb = bytearray(k)
print kb
kb = bytearray("hi")
print kb
devel@moses:~$ python -b test_py.py 
 
hi
devel@moses:~$ python -bb test_py.py 
 
hi

Please correct me if I'm wrong anywhere here. I'd like to work on this.
msg207355 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-01-05 07:12
Try this to trigger a warning:

python2 -b -c 'bytearray("3") == u"3"'
-c:1: BytesWarning: Comparison between bytearray and string
msg219982 - (view) Author: Emily Zhao (emily.zhao) * Date: 2014-06-07 21:38
Might be worth making this addition from 3 (I'm not sure how to add this to 2)
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)


Building on Martin's example:

On all of these, python2 is
Python 2.7.6 (default, Apr  6 2014, 23:14:26)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.


emily-mba:cpython emily$ python2
>>> bytearray("3") == u"3"
False

emily-mba:cpython emily$ python2 -b
>>> bytearray("3") == u"3"
__main__:1: BytesWarning: Comparison between bytearray and string
False

emily-mba:cpython emily$ python2 -bb
>>> bytearray("3") == u"3"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BytesWarning: Comparison between bytearray and string
msg219985 - (view) Author: Emily Zhao (emily.zhao) * Date: 2014-06-07 21:59
Here's an attempt (based on 3's main.c http://hg.python.org/cpython/file/8866ac6f2269/Modules/main.c)
msg220000 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-06-07 23:43
Trouble is, in Python 2 bytes() and str() are the same thing, and most of those conditions don’t apply. Maybe something like this is more correct:

-b     : issue warnings about comparing bytearray with unicode.
         (-bb: issue errors)
msg279704 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-10-30 05:16
As well as the usage string, it is missing from the list of options in the main documentation at Doc/using/cmdline.rst. There are a couple of places (sys.rst and warnings.rst) that reference :option:`-b`, and newer versions of Sphinx complain about this.
msg285272 - (view) Author: Greg Bengeult (gbengeult) * Date: 2017-01-11 21:17
I have attached a patch for 2.7 that adds -b and -bb to the command line documentation in Modules/main.c and Doc/library/cmdline.rst, following the suggested text provided by Martin.

Interestingly, unicode(), bytearray(), and str() don't seem to be transitive in 2.7.

>>> u"3" == str(3)
True

>>> str(3) == bytearray("3")
True

>>> u"3" == bytearray("3")
False
msg285283 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-01-12 03:06
Right, the lack of transitivity comes from the fact that:

    >>> u"3" == str(3)
    True

Is really shorthand for:

    >>> u"3" == str(3).decode("ascii")
    True

However, the implicit decoding only triggers for *exactly* bytes instances, so in the bytearray case it needs to be explicit:

    >>> u"3" == bytearray(b"3").decode("ascii")
    True
msg285284 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-01-12 03:26
Added some review comments on the patch. The only critical one was changing a :class:`bytes` reference to :class:`unicode`, but I also suggested it may be worth mentioning that it *doesn't* enable warnings about all binary/text comparisons the way the Python 3 one does.
msg285334 - (view) Author: Greg Bengeult (gbengeult) * Date: 2017-01-12 17:13
Thanks for the :class: markup edits. I'm still learning this stuff.
msg293612 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-05-13 14:30
New changeset f6c6d1e2304a06efdf65c60c3e1c6bbe1f192fd0 by Zachary Ware in branch '2.7':
bpo-11681: Document the `-b` and `-bb` options (GH-1562)
https://github.com/python/cpython/commit/f6c6d1e2304a06efdf65c60c3e1c6bbe1f192fd0
msg293613 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-05-13 14:46
Thanks for the patch, Greg!  I implemented Nick's suggestions and merged it.
History
Date User Action Args
2017-05-13 14:46:56zach.waresetstatus: open -> closed
resolution: fixed
messages: + msg293613

stage: needs patch -> resolved
2017-05-13 14:30:22zach.waresetnosy: + zach.ware
messages: + msg293612
2017-05-12 21:01:20zach.waresetpull_requests: + pull_request1660
2017-01-12 17:13:54gbengeultsetmessages: + msg285334
2017-01-12 03:26:37ncoghlansetmessages: + msg285284
2017-01-12 03:06:53ncoghlansetmessages: + msg285283
2017-01-11 21:17:04gbengeultsetfiles: + b_option.patch
nosy: + gbengeult
messages: + msg285272

2016-10-30 05:16:49martin.pantersetmessages: + msg279704
2014-06-08 22:17:01Arfreversetnosy: + Arfrever
2014-06-07 23:43:16martin.pantersetmessages: + msg220000
2014-06-07 21:59:05emily.zhaosetfiles: + issue11681.patch
keywords: + patch
messages: + msg219985
2014-06-07 21:38:25emily.zhaosetnosy: + emily.zhao
messages: + msg219982
2014-01-05 07:12:51martin.pantersetnosy: + martin.panter
messages: + msg207355
2014-01-05 05:30:01moijes12setnosy: - moijes12
2012-06-26 18:26:22moijes12setnosy: + moijes12
messages: + msg164089
2011-03-27 18:34:09lemburgsetnosy: + lemburg
messages: + msg132354
2011-03-27 09:57:29ncoghlansetnosy: + ncoghlan
messages: + msg132300
2011-03-26 19:08:02ezio.melottisetnosy: + ezio.melotti
2011-03-26 01:17:56merwoksetmessages: + msg132201
2011-03-26 01:12:00r.david.murraysetmessages: + msg132200
2011-03-26 01:08:34merwoksetnosy: + georg.brandl
messages: + msg132198
2011-03-26 01:05:40r.david.murraysetnosy: + r.david.murray
messages: + msg132197
2011-03-26 00:53:45merwokcreate