Python 3.7.0a0 (default:be70d64bbf88, Dec  1 2016, 21:21:25)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from array import array
>>> a = array('B', [1, 2])
>>> b'%b' % a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: %b requires bytes, or an object that implements __bytes__, not 'array.array'
>>> m = memoryview(a)
>>> b'%b' % m
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: %b requires bytes, or an object that implements __bytes__, not 'memoryview'

Accorfing to documentation [1] objects that follow the buffer protocol should be supported.  Both array.array and memoryview follow the buffer protocol.

[1]: https://docs.python.org/3/library/stdtypes.html#printf-style-bytes-formatting


See also issue 20284 and PEP 461.

printf-style bytes formatting was added mainly for increasing compatibility with Python 2. It was restricted to support mostly features existing in Python 2.

'%s' formatting in Python 3 supports bytes-like objects partially:

>>> b'%s' % array('B', [1, 2])
"array('B', [1, 2])"
>>> b'%s' % buffer(array('B', [1, 2]))
'\x01\x02'
>>> b'%s' % memoryview(array('B', [1, 2]))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot make memory view because object does not have the buffer interface
>>> b'%s' % bytearray(b'abc')
'abc'
>>> b'%s' % buffer(bytearray(b'abc'))
'abc'
>>> b'%s' % memoryview(bytearray(b'abc'))
'<memory at 0xb70902ac>'

I don't know whether there is a need of supporting the buffer protocol in printf-style bytes formatting. bytearray is already supported, buffer() doesn't exist in Python 3, memoryview() is not supported in Python 2. Seems this doesn't add anything for increasing the compatibility.

Isn't this a discussed behaviour that is explicitly documented in PEP 461?

For '%b', it looks like the PEP supports it. I didn't follow the PEP discussions, I think Ethan will know more.

What's your opinions Alexander and Ethan?

Sometimes the implementation can expose drawbacks of initial design. I don't know whether there was good reason for omitting the support of the buffer protocol (in that case the PEP should be updated) or this is just an oversign. We should ask Ethan about this.

The change proposed by Xiang looks correct, but not very efficient. It makes one redundant copy of the data. More efficient implementation will complicate the code, and that can hit the performance of other cases.

I suspect it was a simple oversight, and should be added now.  Since it's been missing for so long I think we should put it in 3.7, maybe put it in 3.6 (maybe not, since it has two point releases out now), but definitely not in 3.5.

@xiang.zhang - I am the OP for this issue, so naturally I expect this to be fixed.  I have a work-around in place for my own code, so I have no opinion on the particular versions.  I guess the normal policy on bug fixes should apply.

Following example copies the entire buffer object while copying only smart part is needed:

    m = memoryview(b'x'*10**6)
    b'%.100b' % m

I don't know whether this is important use case that is worth an optimization. The workaround is using slicing rather than truncating in format:

    b'%b' % m[:100]

Or in the case of general buffer object:

    b'%b' % memoryview(m).cast('B')[:100]

But in that case it is not hard to add an explicit conversion to bytes.

    b'%b' % bytes(memoryview(m).cast('B')[:100])

I committed the suboptimal patch. I close this issue now and if there is any enhancement solution, let's make it another issue. Thank you all.

New changeset faa2cc63e45bc7d7ffab84bebe5a9f4fe065bd96 by Xiang Zhang in branch '3.6':
bpo-28856: Let %b format for bytes support objects that follow the buffer protocol  (GH-664)
https://github.com/python/cpython/commit/faa2cc63e45bc7d7ffab84bebe5a9f4fe065bd96

New changeset 7e2a54cdd977078b40b82182e46b201f8163f659 by Xiang Zhang in branch 'master':
bpo-28856: Let %b format for bytes support objects that follow the buffer protocol (GH-546)
https://github.com/python/cpython/commit/7e2a54cdd977078b40b82182e46b201f8163f659