http://docs.python.org/dev/faq/design.html#why-are-floating-point-calculations-so-inaccurate

This whole paragraph is wrong since #9337 (Make float.__str__ behave identically to float.__repr__).
"""
The str() function prints fewer digits and this often results in the more sensible number that was probably intended:
>>> 1.1 - 0.9
0.20000000000000007
>>> print(1.1 - 0.9)
0.2
"""

Applies from 3.2 onwards.

Agreed.  Do you have a suggested rewrite?

That faq entry could be improved significantly in many ways;  the tutorial material is much better.  I'm wondering whether it would be better to cut down the FAQ to much something much briefer, and leave the link to the tutorial for people who want more.

Proposed rewrite:




Why are floating point calculations inaccurate?
-----------------------------------------------

Users are often surprised by results like this::

   >>> 1.2 - 1.0
   0.199999999999999996

and think it is a bug in Python.  It's not.  This has little to do with Python,
and much more to do with how the underlying platform handles floating-point
numbers.

Python floats are stored internally in binary floating-point, using a fixed
precision (typically 53 bits).  Many numbers that can be written easily in
decimal notation (``1.2``, for example), cannot be expressed exactly in this
internal binary format.  After::

   >>> x = 1.2

the value stored for x is a (very good) approximation to the value ``1.2``, but
is not exactly equal to it.  (On a typical machine, the actual stored value
is::

   1.1999999999999999555910790149937383830547332763671875

which is accurate to around 16 decimal digits.)  Similarly, the result of any
floating-point operation must often be rounded to fit into the internal format,
resulting in another tiny error.

For a more detailed explanation of what's involved, please see the chapter on
:ref:`floating point arithmetic <tut-fp-issues>` in the Python tutorial.

+1

On 03/10/2012 12:26 PM, Mark Dickinson wrote:
>
> Mark Dickinson<dickinsm@gmail.com>  added the comment:
>
> Proposed rewrite:

Hi,
thanks for the quick reply. If we were to rewrite the whole entry, some 
more changes could be done:

I think it would be useful to mention explicitly that Python simply uses 
the native floating-point implementation in hardware and thus behaves 
very similarly to other languages which do this, for instance C or Java. 
This should clear up a lot of the behaviour for people who know other 
programming languages. "how the underlying platform handles 
floating-point" says something very similar, but the reader needs to 
understand what the "underlying platform" exactly is.

It is easy to count, that exactly 17 digits are accurate.

I have to admit, that I'm completely lost here --- why would a vastly 
inaccurate number (with more than half of digits wrong) be ever stored?
If "1.2" is converted to a float (a C double in current implementation), 
it has 15.96 decimal digits of precision.

"Similarly, the result of a floating-point operation must be rounded to 
fit into the fixed precision, often resulting in another tiny error." ?

[part mangled by the tracker]

">     1.1999999999999999555910790149937383830547332763671875
">
"> which is accurate to around 16 decimal digits.)

It is easy to count, that exactly 17 digits are accurate.

I have to admit, that I'm completely lost here --- why would a vastly
inaccurate number (with more than half of digits wrong) be ever stored?
If "1.2" is converted to a float (a C double in current implementation),
it has 15.96 decimal digits of precision.

" > Similarly, the result of any
" > floating-point operation must often be rounded to fit into the
" > internal format, resulting in another tiny error.
"Similarly, the result of a floating-point operation must be rounded to 
fit into the fixed precision, often resulting in another tiny error." ?

> I think it would be useful to mention explicitly that Python simply uses 
> the native floating-point implementation in hardware and thus behaves 
> very similarly to other languages which do this, for instance C or Java. 

Agreed that it's useful to say something here to make it obvious that Python's behaving just like most other mainstream programming languages.  Perhaps most straightforward would be to state that CPython just uses C doubles.  (That sidesteps nitpicking about software floating-point :-).

> It is easy to count, that exactly 17 digits are accurate.

Hmm;  that depends a bit on your definitions.  The relative error here is around 3.7e-17, so in this particular case you're getting something between 16 and 17 decimal digits of accuracy.  Other cases would produce different relative errors, with a max relative error for normal cases of 2**-53, or approximately 1.1e-16.

> I have to admit, that I'm completely lost here --- why would a vastly 
> inaccurate number (with more than half of digits wrong) be ever stored?

Because that number is the unique closest representable IEEE 754 double to the target number (1.2).  You can go through all the steps here:

 - convert 1.2 to binary, to get:

    1.001100110011001100110011.....  repeated ad infinitum

 - round to the nearest 53-bit number:

    1.0011001100110011001100110011001100110011001100110011

 - convert back to decimal to get the number that I gave.

But this sort of detail is already there in the tutorial, and doesn't seem appropriate for the FAQ entry.  For the FAQ, maybe it would be less confusing to give only about a 20-digit approximation.

> "Similarly, the result of any
> floating-point operation must often be rounded to fit into the 
> internal format,
> resulting in another tiny error." ?

The aim here was to make the point that it's not just conversion from decimal to binary that introduces errors;  it's also true that any arithmetic operation has the potential to introduce errors.  For example, 10**16 and 1 are both exactly representable as floats, but the sum of 10**16 + 1 is not;  the result has to be rounded to fit into a float.  Suggestions for rewording to make this clearer are welcome!

> I think it would be useful to mention explicitly that Python simply uses
> the native floating-point implementation in hardware and thus behaves
> very similarly to other languages which do this, for instance C or Java.
> This should clear up a lot of the behaviour for people who know other
> programming languages. "how the underlying platform handles
> floating-point" says something very similar, but the reader needs to
> understand what the "underlying platform" exactly is.

Well, people may just be as confused about the term "native 
implementation" as they are seemingly confused about "underlying
platform".

 >
> I have to admit, that I'm completely lost here --- why would a vastly
> inaccurate number (with more than half of digits wrong) be ever stored?
> If "1.2" is converted to a float (a C double in current implementation),
> it has 15.96 decimal digits of precision.

Since it has this number of decimal digits of precision, the text says
"about 16", rather than "exactly 17", which would mislead people into
thinking that they get 17 digits of precision.

I don't understand your confusion. The numbers are stored in binary,
and it's *not* the case that half of the digits are wrong. In binary,
all 53 bits of mantissa are correct. When displaying this number in
decimal (which is not the way in which it is stored), then you many more
decimal digits to make the decimal representation correct.

Proposed rewrite (building on Mark's version):
- mention C doubles
- explain ``1.2`` example a bit more

"""
Why are floating-point calculations inaccurate?
-----------------------------------------------

Users are often surprised by results like this::

   >>> 1.2 - 1.0
   0.199999999999999996

and think it is a bug in Python.  It's not.  This has little to do with Python,
and much more to do with how the underlying platform handles floating-point
numbers.

The float type in CPython simply uses C ``double`` for storage. The number is stored in binary floating-point with a fixed precision (typically 53 bits) and Python uses C operations, which in turn rely on the hardware implementation in the processor, to perform floating-point operations. This means that as far as floating-point operations are concerned, Python behaves like many popular languages including C and Java.

Many numbers that can be written easily in decimal notation (``1.2``, for example), cannot be expressed exactly in the binary format.  After::

   >>> x = 1.2

the value stored for ``x`` is a (very good) approximation of the value ``1.2``, but
is not exactly equal to it.  On a typical machine, the actual stored value
is::

   1.0011001100110011001100110011001100110011001100110011 (binary)

which is approximately::

   1.19999999999999995559 (decimal)

53 binary digits are equivalent to about 16 decimal digits and in this case the first 17 digits are correct after the conversion back to decimal.

Similarly, the result of any floating-point operation must be rounded to 
fit into the fixed precision, often resulting in another tiny error.

For a more detailed explanation of what's involved, please see the chapter on
:ref:`floating point arithmetic <tut-fp-issues>` in the Python tutorial.
"""

> - explain ``1.2`` example a bit more

+1 for giving the binary representation; -1 for rounding
the decimal version of it.

On 03/10/2012 07:52 PM, Martin v. Löwis wrote:
>> - explain ``1.2`` example a bit more
>
> +1 for giving the binary representation; -1 for rounding
> the decimal version of it.

Oh, 1.1999999999999999555910790149937383830547332763671875
is exactly equal to 
1.0011001100110011001100110011001100110011001100110011. In this case 
yes, rounding it is not beneficial.

So one more version (without this rounding):

"""
Why are floating-point calculations inaccurate?
-----------------------------------------------

Users are often surprised by results like this::

    >>> 1.2 - 1.0
    0.199999999999999996

and think it is a bug in Python.  It's not.  This has little to do with 
Python, and much more to do with how the underlying platform handles 
floating-point numbers.

The float type in CPython simply uses C ``double`` for storage. The 
number is stored in binary floating-point with a fixed precision 
(typically 53 bits) and Python uses C operations, which in turn rely on 
the hardware implementation in the processor, to perform floating-point 
operations. This means that as far as floating-point operations are 
concerned, Python behaves like many popular languages including C and Java.

Many numbers that can be written easily in decimal notation (``1.2``, 
for example), cannot be expressed exactly in the binary format.  After::

    >>> x = 1.2

the value stored for ``x`` is a (very good) approximation of the value 
``1.2``, but is not exactly equal to it.  On a typical machine, the 
actual stored value is::

    1.0011001100110011001100110011001100110011001100110011 (binary)

which is exactly::

    1.1999999999999999555910790149937383830547332763671875 (decimal)

53 binary digits are equivalent to about 16 decimal digits and in this 
case the first 17 digits are correct after the conversion back to decimal.

Similarly, the result of any floating-point operation must be rounded to 
fit into the fixed precision, often resulting in another tiny error.

For a more detailed explanation of what's involved, please see the 
chapter on :ref:`floating point arithmetic <tut-fp-issues>` in the 
Python tutorial.
"""

The last version in the message above looks good to me. Ready to markup and apply?

Fine by me.

Language nitpick:  "approximation to" sounds a bit better to my ears than "approximation of"

Both appear to be commonly used. Both sound OK to me.

Zbyszek, have you signed a contributor agreement form? [1]

If not, please could you do so?  Then I can apply this doc contribution.

Thanks!


[1] http://www.python.org/psf/contrib/

Done now.

Thanks,
Zbyszek

New changeset 2b2a7861255d by Mark Dickinson in branch '3.2':
Issue #14245: Improve floating-point entry in FAQ.  Thanks Zbyszek Jędrzejewski-Szmek for some of the wording.
http://hg.python.org/cpython/rev/2b2a7861255d

New changeset a79b07e05d0d by Mark Dickinson in branch 'default':
Issue #14245: Merge changes from 3.2.
http://hg.python.org/cpython/rev/a79b07e05d0d

Thanks for all the feedback.  I made another round of minor edits to the latest version and committed the result.