Let bin() show floating point values. This would contribute quite a
bit to people's understanding of floating point arithmetic. It has a
nice education value and it makes it easier to diagnose floating point
mysteries.

def vis(f):
    """ Show binary representation of a floating point number:

        >>> vis(math.pi)
        '0b11.001001000011111101101010100010001000010110100011'
        >>> vis(-0.375)
        '-0b0.011'
    """
    f, sign = (f, '') if f >= 0 else (-f, '-')
    n, d = f.as_integer_ratio() if isinstance(f, float) else (f, 1)
    n, d = map(lambda x: bin(x)[2:], (n, d))
    n = n.rjust(len(d), '0')
    s = list(n)
    s.insert(len(n) - len(d) + 1, '.')
    return sign + '0b' + ''.join(s)

I'm not sure about the educational value of letting obscure bugs
creeping in when someone passes a float where an int is expected :-)
Your vis() function looks appropriate for the task of educating people,
why should its behaviour be folded into bin()?

Or, if you want to educate people at all cost, the TypeError raised by
bin() when trying it on a float object could include in its message the
binary representation of the float object "for edification". :-)

I like the idea in general. It is way too common for people to be
confused by decimal representation of floats, so an obvious mechanism to
display the true value will be helpful.

I don't think a promiscuous bin() will lead to obscure bugs. Antoine
will have to give a specific example to convince me otherwise. I can
see, that extending :b format to floats may be a problem, but this is
not being proposed.

As far as the proposed implementation goes, I don't like the fact that
it always returns a fixed rather than floating point notation. (Try
vis(1e100)). I think it should behave somewhat like "%g" format.

If bin() is extended to floats, it may become confusing that hex() and
oct() are not similarly extended.

Well it's quite simple. Imagine you have a function f() which takes an
integer parameter named x, and somewhere applies bin() to this parameters.

Right now, if you call f(1.0) instead of f(1), you will get a TypeError,
which is easy to detect: you then fix the call to f(1), and bin()
produces the expected result ('0b1').

With Raymond's suggestion, if you call f(1.0) instead of f(1), no
exception will tell you your mistake, and bin() will produce a
completely bogus result compared to the expected one. If you notice the
bogus function output and find out that it contains a strange-looking
string (something like
'0b11.001001000011111101101010100010001000010110100011'), it is still
not obvious from it that the problem stems from passing a float instead
of an int. Especially if f() is a library function which you didn't
write yourself.

There is a reason Python recently introduced a stronger distinction
between ints and floats (for instance the __index__ method, which bin()
seems to use currently), I don't see the logic behind trying to undo it.

And it's not like printing the bin() representation of a float has any
actually use (besides education, but education can use its own tools
rather than builtin functions).

On Fri, May 30, 2008 at 10:52 AM, Antoine Pitrou <report@bugs.python.org> wrote:

Antoine Pitrou <pitrou@free.fr> added the comment:

Well it's quite simple. Imagine you have a function f() which takes an
integer parameter named x, and somewhere applies bin() to this parameters.

This is too abstract. Depending on what f() is designed to do,
accepting floats may or may not be the right thing. For example, if
bin is used inside f() only to produce some log output, but
otherwise f() works on any number, promiscuous bin() will actually
make an application using f() more robust.

Right now, if you call f(1.0) instead of f(1), you will get a TypeError,
which is easy to detect: you then fix the call to f(1), and bin()
produces the expected result ('0b1').

There is no "right now". Builtin bin is new in 2.6.

..

There is a reason Python recently introduced a stronger distinction
between ints and floats (for instance the __index__ method, which bin()
seems to use currently), I don't see the logic behind trying to undo it.

I think you are mistaken. Python always distinguished between floats
and integers

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: list indices must be integers
>>> int.__index__
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: type object 'int' has no attribute '__index__'

__index__ was introduced in order to allow user-defined class
instances to be used as indices without affecting the way floats are
treated.

And it's not like printing the bin() representation of a float has any
actually use (besides education, but education can use its own tools
rather than builtin functions).

That's exactly my point. Builtin bin being new, I cannot comment on
its actual use, but I've only used hex() in interactive sessions as an
easy to type alternative to the finger-twisting "0x%x" % incantation.
(IMO, "{0:b}".format(..) is even worse.) In the scripts, however, you
would rarely need to create a string representation of single number.
More often you will need to embed a number in a larger message and
thus you will use % or str.format instead of bin/hex/oct anyways.

BTW, google code search quickly revealed the following antipattern:

   log.msg("real checksum: %s"%hex(hdr[19]))

(twisted-Zope-3.2.2/twisted/words/protocols/toc.py)

Surely "real checksum: %x" % hdr[19] would be a better choice. (The
0x prefix generated by hex is just noise in the log output.)

While writing my previous comments I did not realize that '%x' % accepts
floats:

>>> "%x" % 3.1415
'3'

Float support has been even improved somewhat since 2.5:

Python 2.5 (r25:51908, Nov 24 2006, 11:03:50) 
>>> '%x' % 1e10
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: int argument required

The new stile formatting, however does not allow floats with either :x
or :b formats:

>>> "{0:x}".format(1.)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown conversion type x

>>> "{0:b}".format(1.)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown conversion type b

I don't think anything needs to be done about it given that % formatting
is being deprecated while new style format is doing the right thing IMO.

I think you are mistaken. Python always distinguished between floats
and integers

Sorry, my bad.

For example, if
bin is used inside f() only to produce some log output, but
otherwise f() works on any number, promiscuous bin() will actually
make an application using f() more robust.

First I'm not sure bizarre log output should be considered "more
robust". Second if you are outputting the bin() of an integer as part of
a log output, it probably means the integer should be understood as a
kind of bitfield, and then I don't see how accepting float values is
"more robust" (again). Anyway Python doesn't accept floats for bit-wise
operators (e.g. "1.0&128" raises TypeError).

I still find this proposal undesirable, for several reasons:

1. while taking the binary representation of an integer has a real
   meaning, taking the binary representation of a float only exposes an
   implementation detail, that is the internal layout of float numbers.

2. if two numbers (one integer and one float) are equal, it sounds
   expectable that calling a function on them will produce similar output
   (or fail). Of course, this is not always the case, str(1) and str(1.0)
   are different. But they are not wildly different and the difference is
   still easily understood. While getting something like
   '0b11.001001000011111101101010100010001000010110100011' while you were
   waiting for '0b11' is disturbing.

3. I'm skeptical about the educational value. People who don't know
   about the internal layout of float numbers won't very likely feel
   enlightened by a string of 1s and 0s. Showing a bunch of bits does not
   really explain a structure.

One related feature, though, would be to know whether a string
representation of a float is exact or not.
If we allowed repr() to lose its round-trippability, this could be
implemented by making repr(0.5) return "0.5 (exact)" and repr(0.1)
return "0.10000000000000001 (inexact)".
Or this could be a dedicated method.

While writing my previous comments I did not realize that '%x' %
accepts floats:

And witness how it does something rather intuitive (convert the argument
to int) rather than show the internal layout of the float number in
hexadecimal form :-)

On Fri, May 30, 2008 at 1:47 PM, Antoine Pitrou <report@bugs.python.org> wrote:
..

2. if two numbers (one integer and one float) are equal, it sounds
   expectable that calling a function on them will produce similar output
   (or fail). .. While getting something like
   '0b11.001001000011111101101010100010001000010110100011' while you were
   waiting for '0b11' is disturbing.

I fail to see how the proposed bin(..) can produce '0b11.00100..' from
a float that compares equal to 3.

On Fri, May 30, 2008 at 1:47 PM, Antoine Pitrou <report@bugs.python.org> wrote:
..

1. while taking the binary representation of an integer has a real
   meaning, taking the binary representation of a float only exposes an
   implementation detail, that is the internal layout of float numbers.

You may have misunderstood the proposal. I read the proposal as
producing the true mathematical radix 2 representation of a float
rather than its 64-bit memory layout.

What would you say to adding float-capable bin/oct/hex (+ maybe tobase)
to the math module?

I read the proposal as
producing the true mathematical radix 2 representation of a float
rather than its 64-bit memory layout.

The term "layout" was probably badly chosen.
Still, the explicit motivation for producing that representation is that
it it supposed to educate people about the actual implementation of
floats. Other than that, a radix 2 representation is quite an obscure
(and almost never used) way of representing float objects. Binary
representation of integers in comparison is more widely understood and
more frequently used, which -- I suppose -- justifies the existence of a
builtin function to obtain it.

I fail to see how the proposed bin(..) can produce '0b11.00100..' from
a float that compares equal to 3.

Oops, you are right. bin(3.0) would produce '0b11.', which is indeed
more readable.

What would you say to adding float-capable bin/oct/hex (+ maybe tobase)
to the math module?

Why not indeed.
However, as I said, if the intent is float-specific (and it is, unless
there is another inexact numeric type which warrants a specific bin()
output), then why not make it a method of float?

Better to just build-out bin() and be done with it.

FWIW, the action on ints and floats is similar to how str() handles
numeric inputs:
str(3) --> '3'
str(3.0) --> '3.0'
bin(3) --> '0b11'
bin(3.0) --> '0b11.0'

Better to just build-out bin() and be done with it.

Ok, someone has to take a decision anyway.

However, if you do that, it should be probably decided first what
happens for hex() and oct(). Do they still disallow floats (which is
semantically reasonable but makes things slightly inconsistent)? Or are
they also changed in order to return the same kind of things as bin()
does (which would probably give awful results)?

AFAICT, there is no good use case for showing floats in in hex or oct,
so those should not chance. Also, since bin() is new, no existing code
is affected. Thanks for the comments.

Another problem with bin() on floats is that it will be a one-way street
because as far as I know, there is no function to convert a binary
string back to a float.

My last thought on this issue is that it will be helpful to add
tobase(f, n) and frombase(s,n) functions to math module (and I withdraw
my suggestion of using %g-like notation due to the conflict between E
for exponent and E for 0xE). On the other hand, there is no need to
mess up with builtin bin/oct/hex because with math.tobase float-capable
extensions will be trivially implementable.

On Fri, May 30, 2008 at 2:32 PM, Antoine Pitrou <report@bugs.python.org> wrote:

then why not make it a method of float?

.. because you rarely want to make your functions accept 1.0, but
reject 1 and using f.bin() in your function will give it this
property.

AFAICT, there is no good use case for showing floats in in hex

It is my impression that hexadecimal is more common than binary, in the
numerical analysis community, for exact representation of floats.

For example:
http://hal.archives-ouvertes.fr/docs/00/28/14/29/PDF/floating-point-article.pdf

"Hexadecimal floating-point representations are especially important
when values must be represented exactly, for reproducible results — for
instance, for testing “borderline cases” in algorithms."

Or course, without hex float literals or an equivalent function for
input, hex float output is not much use.

No need to conquer the whole world. Start with bin(1.1). If requests
pour-in for hex(1.1) or for a reverse routine, we can deal with those
when we get them (in the context of their use cases).

Kind of a cool hack. I am not sure how useful it is though. Personally,
I find struct.pack('d', f) slightly more educative (more useful
too), but that is just me.

And if you allow bin() to accept floats, will you make Python accept
binary float literals too?

Saving this for after the first beta goes out.

To address the ideas brought-up so far, here's a new version that can
work with eval. The same appoach extends to oct and hex as well:

def newbin(f):
    """
        >>> newbin(3.125)
        '0b11001 * 2.0 ** -3'
    """
    n, d = f.as_integer_ratio()
    s = '%s * 2.0 ** %d' % (bin(n), -math.log(d, 2.0))
    return s

The other reviewers asked for:

• same treatment for oct() and hex()
• platform independent exact representation of floats
• fixed-size exponential style output instead of tons of leading zeros
• output that round-trips through eval()
• use Py2.6 octal format in 2.6, and 3.0 format in 3.0

Attaching a patch with tests.

I don't like this modification of a PyString object:

+ n = PyString_GET_SIZE(conv);
+ conv_str = PyString_AS_STRING(conv);
+ /* Remove the trailing 'L' if present */
+ if (n && conv_str[n-1] == 'L')
+ conv_str[n-1] = '\0';
+ result = PyString_FromFormat("%s * 2.0 ** %d", conv_str, exp);

The string may have other references (ex: all single-char strings are
shared) and it seems unwise to directly modify the memory.

Also, tests should check if negative numbers have the same
representation as their absolute value (with the sign). It is not
obvious from the implementation, which uses floor().

The patch looks good to me.

It's a bit unfortunate that -0.0 doesn't round-trip correctly (the sign
of the zero gets lost):

>>> eval(bin(-0.0))
0.0

I don't know whether it's worth special-casing this; the output would
have to be in a different format: '-0b0 * 2.0 ** 0' isn't good enough,
since unary minus has higher precedence than multiplication.

Hmm, I don't see a way to preserve -0.0 without varying from the
standard format.

Attaching an updated patch for Amaury's comments.

...and the tests for hex_float.py

I'm looking forward to your C implementation.

Raymond, Mark? Is a new patch with tests and docs forthcoming? Have
you decided on the API yet? I'm willing to approve this for beta 2,
which will be around July 15.

Mark, I'm tied-up with EuroPython until the 14th. Do you have time to
take a crack at this?

I'm working on it. I expect to have something ready by the end of this
weekend.

BTW couldn't you use the %a feature built into C99 to implement this?
(Both input and output?)

Sure. What about non-C99 machines? I thought Python code was only
allowed to assume ANSI C.

On Fri, Jul 4, 2008 at 7:12 AM, Mark Dickinson <report@bugs.python.org> wrote:

Mark Dickinson <dickinsm@gmail.com> added the comment:

Sure. What about non-C99 machines? I thought Python code was only
allowed to assume ANSI C.

I'm pretty sure that's a thing of the past. Ask MvL.

Microsoft compilers implement %a since VS8.0.
VS7.1 does not have it.

In the interests of getting early feedback, here's half a patch,
containing an implementation of from.fromhex and tests.

Still to come: float.hex and documentation.

I'll ask on python-dev about C99 and %a.

containing an implementation of from.fromhex and tests.

That should be 'float.fromhex', not 'from.fromhex'.
I should also have said that this patch is against the trunk; only minor
changes should be required for py3k.

Here's an updated patch, complete with both float methods and
documentation.

Add updated patch with expanded documentation.

Here's a slightly more polished version of the previous patch; no
behaviour changes.

Let me know if there's anything I can do to help get this in before next
week's beta. Anybody want to trade patch reviews?

Minor modifications to the previous patch, mostly to the docs.

Setting priority to critical, since this really needs to go in before the
next beta if it's going to get into 2.6/3.0.

Here's an updated patch that makes the trailing 'p123' exponent optional
in fromhex. (This matches the behaviour of C99's strtod and sscanf; in
contrast, Java always requires the exponent.)

I'm beginning to wonder whether the '0x' shouldn't also be optional on
input as well, in the same way that it's optional in int():

>>> int('0x45', 16)
69
>>> int('45', 16)
69

This would then allow, e.g.,

>>> float.fromhex('45')
69.0

In the spirit of being "liberal in what you accept, but strict in what you
emit", here's a version that makes both the leading '0x' and the trailing
'p...' exponent optional on input. Both of these are still produced on
output.

Note that this version is still perfectly interoperable with C99 and Java
1.5+: fromhex accepts anything produced by C and Java (e.g. via C's '%a',
or Java's toHexString), and the output of hex can be read by C99's
strtod/sscanf and Java's Double constructor, and/or used as hex literals
in C or Java source.

Some final tinkering:

• docstrings and docs expanded slightly; docs mention interoperability
  with C and Java.

• in float.hex(), there's always a sign included in the exponent (e.g.
  "0x1p+0" instead of "0x1p0"). This just makes for a little bit more
  consistency with repr(float), with C99 and with the way the Decimal module
  behaves (but not with Java, which omits the + sign).

So far this looks good. Will complete the review on the next leg of my
flight (about 12 hrs).

The patch looks good. I would coded hex_from_char() using a lookup
into "0123456789abcdef" which uses no unpredicatable branches.
Likewise, I would done hex_from_char() with a case statement (limiting
the call to single unpredicatable branch).

Question: are the ("0x0p+0") and ("-0x0p+0") special cases standard?

The docs need a "new in py2.6"

Coding style: move the inner si++ to a separate line so there are no
side-effects and the order of execution is obvious.

Question: should the "inf" string checks be made case sensitive on
insensitive? Is there a standard?

Here's an updated patch that addresses Raymond's concerns.

The patch looks good. I would coded hex_from_char() using a lookup
into "0123456789abcdef" which uses no unpredicatable branches.
Likewise, I would done hex_from_char() with a case statement (limiting
the call to single unpredicatable branch).

Done.

Question: are the ("0x0p+0") and ("-0x0p+0") special cases standard?

Not entirely. Java outputs "0x0.0p0" and "-0x0.0p0". The C99 standard
doesn't specify exactly how the output should look, except to say that
the exponent should be 0. The '+' is there for consistency.

I can change '0x0p+0' to '0x0.0p+0' if people think this looks prettier.
On consideration, this does look better to me. Changed.

The docs need a "new in py2.6"

Fixed.

Coding style: move the inner si++ to a separate line so there are no
side-effects and the order of execution is obvious.

Done. (And the same with s++ in float_fromhex.)

Question: should the "inf" string checks be made case sensitive on
insensitive? Is there a standard?

Everything I've seen, except Java, seems to like case-insensitivity.
The C99 standard says case should be ignored, as does the IBM Decimal
standard. Python's float('nan') and float('inf') also currently ignore
case. So I think these checks should be case insensitive.

(Java insists on infinity being spelt "Infinity", and nan being spelt
"NaN".)

Thank you for reviewing this, Raymond!

I aim to check this in when (if?) I get approval from Barry.

If you two can agree that this code is good, I'm ok with the API.

I would emphasize in the docs and NEWS entry though that .hex() is an
*instance* method while .fromhex() is a *class* method.

Mark, please go ahead and apply so the buildbots will have time to give
it a run on all the platforms before beta 2 is cut. Be sure to make
Guido's edits to the Misc/NEWS entry.

Committed, r64974