classification
Title: Add examples to int.to_bytes and int.from_bytes
Type: enhancement Stage: needs patch
Components: Documentation Versions: Python 3.4, Python 3.3, Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: asvetlov, docs@python, ezio.melotti, mark.dickinson, paddy3118, serhiy.storchaka, woparry
Priority: normal Keywords: easy, patch

Created on 2012-11-29 19:25 by paddy3118, last changed 2013-05-26 10:35 by mark.dickinson.

Files
File name Uploaded Description Edit
issue16580.patch woparry, 2013-05-25 21:05 review
Messages (9)
msg176671 - (view) Author: Paddy McCarthy (paddy3118) Date: 2012-11-29 19:25
http://docs.python.org/3.3/library/stdtypes.html?highlight=to_bytes#int.to_bytes and http://docs.python.org/3.3/library/stdtypes.html?highlight=to_bytes#int.to_bytes would benefit from an example showing what they do based on simpler coding.

I have such an example that I wrote here: http://paddy3118.blogspot.co.uk/2012/11/some-identities-for-python-inttobytes.html that you can use.

I.e.

>>> n = 2491969579123783355964723219455906992268673266682165637887
>>> length = 25
>>> n2bytesbig    = n.to_bytes(length, 'big')
>>> n2byteslittle = n.to_bytes(length, 'little')
>>> assert n2bytesbig    == bytes( (n >> i*8) & 0xff for i in reversed(range(length)))
>>> assert n2byteslittle == bytes( (n >> i*8) & 0xff for i in range(length))
>>> assert n == sum( n2bytesbig[::-1][i] << i*8 for i in range(length) )
>>> assert n == sum( n2byteslittle[i]    << i*8 for i in range(length) )
>>> assert n == int.from_bytes(n2bytesbig,    byteorder='big')
>>> assert n == int.from_bytes(n2byteslittle, byteorder='little')
>>>
msg177042 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-12-06 14:26
Your example is comprehensive but not simple and obvious. 
I think better to keep it out of doc.
msg177043 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-12-06 14:31
I agree.  The examples in the doc seem clear to me, whereas the ones you proposed are not as clear.  Do you think there's something that they don't currently cover that should be added?
msg177207 - (view) Author: Paddy McCarthy (paddy3118) Date: 2012-12-09 10:34
On 06/12/2012 14:31, Ezio Melotti wrote:
> Ezio Melotti added the comment:
>
> I agree.  The examples in the doc seem clear to me, whereas the ones you proposed are not as clear.  Do you think there's something that they don't currently cover that should be added?
>
> ----------
> nosy: +ezio.melotti
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue16580>
> _______________________________________
>
First, Thanks Ezio and Andrew for your replies.

My problem was that when working on bitcoin address validation I saw 
code that was shifting and &'ing with 0xFF to convert to multiple bytes 
and half remembered that there might be a Python function to do that. On 
finding the .to_bytes method and its parameter "big" or "little", the 
only way I had of working out which to use was to try each until I found 
out which worked.

I therefore thought that what would have helped me was code that showed 
the equivalent "expanded Python" for the method in a similar way to what 
is done for some of the itertools functions etc.

If we split my request into two:

 1. Is such extra explanation necessary.
 2. Is my specific code that extra explanation.

I can work on the code a bit more.
Have I persuaded you that an extra explanation is necessary?

Thanks, Paddy.

P.S. I guess what is currently present shows the result of the methods 
but nothing on how it could be generated. I am stating that the 
generation can aid comprehension.
msg177208 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-12-09 10:55
Usually we add plain Python equivalents when they are simple enough that the code equivalent is as understandable as the prose or more (see for example http://docs.python.org/3/library/functions.html#all, or the itertools functions you mentioned).
For this case I think it would help if you presented an equivalent function, e.g.:

def to_bytes(n, length, order):
    if order == 'little':
        return bytes((n >> i*8) & 0xff for i in range(length))
    elif order == 'big':
        return bytes((n >> i*8) & 0xff for i in reversed(range(length)))

or even:

def to_bytes(n, length, order):
    indexes = range(length) if order == 'little' else reversed(range(length))
    return bytes((n >> i*8) & 0xff for i in indexes)

This is also done for http://docs.python.org/3.3/library/stdtypes.html#int.bit_length just above to/from_bytes, so it might be a good addition.
If this is done, the equivalent function can also be added to the test suite, so we can verify that it's indeed equivalent.
msg177262 - (view) Author: Paddy McCarthy (paddy3118) Date: 2012-12-10 05:57
On 09/12/2012 10:55, Ezio Melotti wrote:
> Ezio Melotti added the comment:
>
> Usually we add plain Python equivalents when they are simple enough that the code equivalent is as understandable as the prose or more (see for example http://docs.python.org/3/library/functions.html#all, or the itertools functions you mentioned).
> For this case I think it would help if you presented an equivalent function, e.g.:
>
> def to_bytes(n, length, order):
>      if order == 'little':
>          return bytes((n >> i*8) & 0xff for i in range(length))
>      elif order == 'big':
>          return bytes((n >> i*8) & 0xff for i in reversed(range(length)))
>
> or even:
>
> def to_bytes(n, length, order):
>      indexes = range(length) if order == 'little' else reversed(range(length))
>      return bytes((n >> i*8) & 0xff for i in indexes)
>
> This is also done for http://docs.python.org/3.3/library/stdtypes.html#int.bit_length just above to/from_bytes, so it might be a good addition.
> If this is done, the equivalent function can also be added to the test suite, so we can verify that it's indeed equivalent.
>
> ----------
> keywords: +easy
> stage:  -> needs patch
> versions: +Python 2.7, Python 3.2, Python 3.4
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue16580>
> _______________________________________
>
The second example looks great. I like the dual use for testing too and 
will try and remember both the next time I find I have ireas about the 
documentation.

Thanks guys. It's appreciated!
msg190004 - (view) Author: W. Owen Parry (woparry) Date: 2013-05-25 21:05
Patch adding examples + tests for equivalence. Comments appreciated.

In particular, I'm not sure that the from_bytes example is simple enough to be useful:

def from_bytes(bytes, byteorder, signed=False):
    if byteorder == 'little':
        little_ordered = list(bytes)
    elif byteorder == 'big':
        little_ordered = list(reversed(bytes))
     n = sum(little_ordered[i] << i*8 for i in range(len(little_ordered)))
    if signed and little_ordered and (little_ordered[-1] & 0x80):
         n -= 1 << 8*len(little_ordered)
    return n
msg190013 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-05-25 21:39
I don't call it the examples. Here is the examples:

>>> (1234).to_bytes(2, 'big')
b'\x04\xd2'
>>> (1234).to_bytes(2, 'little')
b'\xd2\x04'
>>> (-1234).to_bytes(2, 'big', signed=True)
b'\xfb.'
>>> int.from_bytes(b'\xde\xad\xbe\xef', 'big')
3735928559
>>> int.from_bytes(b'\xde\xad\xbe\xef', 'little')
4022250974
>>> int.from_bytes(b'\xde\xad\xbe\xef', 'big', signed=True)
-559038737
msg190071 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-05-26 10:35
The `from_bytes` example code looks fine to me, except that I'd probably use `enumerate` in the iteration;  i.e.,

    sum(b << 8*i for i, b in enumerate(little_ordered))

instead of

    sum(little_ordered[i] << i*8 for i in range(len(little_ordered)))

Also, in:

   if signed and little_ordered and (little_ordered[-1] & 0x80):

I wondered why you needed the `little_ordered` check.  But I see that `int.from_bytes(b'', 'little', signed=True)` produces `0`, which is a little bit disappointing:  I was expecting an exception.  (A signed format should have a sign bit, which is impossible if the length of the byte string is 0.)

The `to_bytes` example code is missing range checks for the input.  It may be clearer to simply state that in the docs, instead of modifying the example code to add the range checks.
History
Date User Action Args
2013-05-26 10:35:19mark.dickinsonsetnosy: + mark.dickinson
messages: + msg190071
2013-05-25 21:39:18serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg190013
2013-05-25 21:05:07woparrysetfiles: + issue16580.patch

nosy: + woparry
messages: + msg190004

keywords: + patch
2013-01-13 16:10:59Ramchandra Aptesettitle: Add examples to int.to_bytres and int.from_bytes -> Add examples to int.to_bytes and int.from_bytes
2012-12-10 05:57:39paddy3118setmessages: + msg177262
2012-12-09 10:55:36ezio.melottisetkeywords: + easy

stage: needs patch
messages: + msg177208
versions: + Python 2.7, Python 3.2, Python 3.4
2012-12-09 10:34:49paddy3118setmessages: + msg177207
2012-12-06 14:31:38ezio.melottisetnosy: + ezio.melotti
messages: + msg177043
2012-12-06 14:26:29asvetlovsetnosy: + asvetlov
messages: + msg177042
2012-11-29 19:25:09paddy3118create