classification
Title: Context.create_decimal_from_float() inconsistent precision for zeros after decimal mark
Type: behavior Stage:
Components: Versions: Python 3.3
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: mark.dickinson, mdealencar, skrah
Priority: normal Keywords:

Created on 2014-02-03 14:28 by mdealencar, last changed 2014-02-04 20:46 by mdealencar. This issue is now closed.

Messages (13)
msg210131 - (view) Author: Mauricio de Alencar (mdealencar) Date: 2014-02-03 14:28
The following code demonstrates an inconsistency of this method in dealing with zeros after the decimal mark.


from decimal import Context

context = Context(prec=2)

for x in [100., 10., 1., 0.1]:
    print(context.create_decimal_from_float(x), context.create_decimal_from_float(4.56*x))


Produces the output:
1.0E+2 4.6E+2
10 46
1 4.6
0.10 0.46

Line 3 is inconsistent. It should be "1.0 4.6".
msg210199 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-02-04 11:02
The output is correct, though the tiny precision makes it look strange.  The decimal module is following the usual rules for 'ideal' exponents:

- For *exactly representable* results, the ideal exponent is 0, and the output will be chosen to have exponent as close to that as possible (while not altering the value of the result).

- Where the result isn't exactly representable, full precision is used.

Those two rules together explain all of the output you showed:  100.0, 10.0 and 1.0 are exactly representable, so we aim for an exponent of 0.  But 100.0 can't be expressed in only 2 digits with an exponent of 0, so it ends up with an exponent of 1, hence the `1.0E+2` output.

460.0 and 46.0 are similarly exactly representable.

4.6 and 0.46 are not exactly representable, so the output is given to full precision.
msg210206 - (view) Author: Mauricio de Alencar (mdealencar) Date: 2014-02-04 11:42
According to the docs (http://docs.python.org/3/library/decimal.html):

"The decimal module incorporates a notion of significant places so that 1.30 + 1.20 is 2.50. The trailing zero is kept to indicate significance. This is the customary presentation for monetary applications. For multiplication, the “schoolbook” approach uses all the figures in the multiplicands. For instance, 1.3 * 1.2 gives 1.56 while 1.30 * 1.20 gives 1.5600."

Therefore, if I request 2 digits of precision, I expect 2 digits in the output.

In addition, the docs assert that "Decimal numbers can be represented exactly", which leaves me lost about you argument on whether some number is *exactly representable* or not.
msg210210 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-02-04 11:56
> Therefore, if I request 2 digits of precision, I expect 2 digits in the output.

The `prec` attribute in the context refers to the total number of *significant digits* that are storable, and not to the number of digits after the decimal point.  `Decimal` is at heart a floating-point type, not a fixed-point one (though the handling of significant zeros that you note means that it's useful in fixed-point contexts too).  For typical uses you'll want `prec` to be much bigger than 2.

So the number of trailing zeros is typically determined not by `prec` but by the exponents of the operands to any given operation.  In the example you cite, the output is `2.50` because the inputs both had two digits after the point.

> the docs assert that "Decimal numbers can be represented exactly"

Sure, but for example the 0.1 in your code is not a Decimal: it's a Python float, represented under the hood in binary.  Its exact value is 0.1000000000000000055511151231257827021181583404541015625, and that can't be stored exactly in a Decimal object with only two digits of precision.

So the behaviour you identify isn't a bug: the module is following a deliberate design choice here.
msg210225 - (view) Author: Mauricio de Alencar (mdealencar) Date: 2014-02-04 13:10
I propose then to create a context setting that switches between the current textual representation of a Decimal and one that is "right zeros padded" up to context precision.

Such that the line:

print(Context(prec=4, precise_repr=True).create_decimal_from_float(1.))

would output "1.000" 



I post bellow a workaround for getting the result I expect just in case there is anybody else with similar expectations.


from decimal import Context

def dec(num, prec):
    return Context(prec=prec).create_decimal('{{:.{:d}e}}'.format(prec - 1).format(num))



Thanks for you attention.
msg210226 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-02-04 13:19
If you're after a particular string representation, you'll probably find that string formatting meets your needs.

Python 3.3.3 (default, Nov 24 2013, 14:34:37) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal
>>> x = Decimal('3.3')
>>> x
Decimal('3.3')
>>> "{:.4f}".format(x)  # string representation with 4 digits after the point.
'3.3000'

Reclosing this issue.  If you want to pursue your proposal, please open a separate issue for that rather than reopening this one.  Thanks!
msg210228 - (view) Author: Mauricio de Alencar (mdealencar) Date: 2014-02-04 13:39
String formatting is completely unaware of the concept of *significant digits*. The only format that can get it right for every case is the 'e'. But then you always get the exponent, which is undesirable. I was hopeful that the decimal module would handle significant digits as I need.

I will settle with the workaround I posted earlier.

Thanks anyway.
msg210229 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-02-04 13:58
Mauricio de Alencar <report@bugs.python.org> wrote:
> String formatting is completely unaware of the concept of *significant digits*.

>>> format(Decimal(1), ".2f")
'1.00'
msg210232 - (view) Author: Mauricio de Alencar (mdealencar) Date: 2014-02-04 14:19
"Digits after the decimal mark" is not the same as "significant digits".
See https://en.wikipedia.org/wiki/Significant_figures

If I have a list of numbers [256.2, 1.3, 0.5] that have 3 significant digits each, I would like to have them displayed as:
['256', '1.30', '0.500']


['{:.2e}'.format(_) for _ in [256.2, 1.3, 0.5]]

would print:

['2.56e+02', '1.30e+00', '5.00e-01']

Which gets the digits right, but is not as desired.

But if I use


from decimal import Context

def dec(num, prec):
    return Context(prec=prec).create_decimal('{{:.{:d}e}}'.format(prec - 1).format(num))

[str(dec(_, 3)) for _ in [256.2, 1.3, 0.5]]


The the output is:
['256', '1.30', '0.500']
msg210237 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-02-04 14:46
Mauricio de Alencar <report@bugs.python.org> wrote:
> 
> Mauricio de Alencar added the comment:
> 
> "Digits after the decimal mark" is not the same as "significant digits".
> See https://en.wikipedia.org/wiki/Significant_figures
> 
> If I have a list of numbers [256.2, 1.3, 0.5] that have 3 significant digits each, I would like to have them displayed as:
> ['256', '1.30', '0.500']

You need to stop lecturing.  The above sentence you wrote directly contradicts
the Wikipedia link you have thrown at us.  And yes, thank you, we do know what
significant figures are.

FYI, the Python implementation of decimal, the C implementation of decimal
and decNumber are completely separate implementations of

    http://speleotrove.com/decimal/decarith.html

by different authors and produce exactly the results that you criticize.
msg210244 - (view) Author: Mauricio de Alencar (mdealencar) Date: 2014-02-04 15:46
> You need to stop lecturing.

I'm sorry, I didn't mean to offend anyone. I just felt I was failing to
communicate the issue when I got the suggestion to use format(Decimal(1),
".2f").

>  The above sentence you wrote directly contradicts the Wikipedia link you
have thrown at us.

The floats I posted are examples of computation results. The meaningful
figures are related to the precision of the measurements fed to the
computation.

> by different authors and produce exactly the results that you criticize.

I understand the design choice for decimal. I just miss a pythonic way of
dealing with significant figures.
msg210254 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-02-04 17:54
Mauricio de Alencar <report@bugs.python.org> wrote:
> The floats I posted are examples of computation results. The meaningful
> figures are related to the precision of the measurements fed to the
> computation.

Thank you, that makes it clear.  Constructing Decimal('256.2') preserves
the exact value, regardless of the context precision.  All arithmetic
functions accept arbitrary precision input and use all digits regardless
of the context precision.

To put it differently, decimal refuses to guess and treats any input
as having the correct number of significant digits.

If you want to attribute a significance to a series of input numbers,
I guess you have to do it manually, using something like:

def mk_full_coeff(x):
    prec = getcontext().prec
    adj = x.adjusted()
    if adj >= prec:
        return +x
    else:
        return x.quantize(Decimal(1).scaleb(adj-prec+1))

>>> c = getcontext()
>>> c.prec = 3
>>> [mk_full_coeff(x) for x in [Decimal('256.2'), Decimal('1.3'), Decimal('0.5')]]
[Decimal('256'), Decimal('1.30'), Decimal('0.500')]
msg210264 - (view) Author: Mauricio de Alencar (mdealencar) Date: 2014-02-04 20:46
Thank you. This function accomplishes what I need, avoiding the
float->string->Decimal conversion path.

I will use a slight variation of it accepting floats and a precision value:

from decimal import Decimal, Contextdef sigdec(f, prec):    x =
Context(prec=prec).create_decimal_from_float(f)    adj = x.adjusted()
  if adj >= prec - 1:        return x    else:        return
x.quantize(Decimal(1).scaleb(adj-prec+1))

Since create_decimal_from_float() applies the precision upon conversion,
the +x trick is not needed. I also noticed that (adj >= prec - 1) does the
job, avoiding the else block in a few more cases.
History
Date User Action Args
2014-02-04 20:46:44mdealencarsetmessages: + msg210264
2014-02-04 17:54:34skrahsetmessages: + msg210254
2014-02-04 15:46:45mdealencarsetmessages: + msg210244
2014-02-04 14:46:50skrahsetmessages: + msg210237
2014-02-04 14:19:44mdealencarsetmessages: + msg210232
2014-02-04 13:58:42skrahsetmessages: + msg210229
2014-02-04 13:39:02mdealencarsetmessages: + msg210228
2014-02-04 13:26:26mark.dickinsonsettitle: Context setting to print Decimal with as many digits as the "prec" setting -> Context.create_decimal_from_float() inconsistent precision for zeros after decimal mark
2014-02-04 13:19:38mark.dickinsonsettype: enhancement -> behavior
versions: + Python 3.3
2014-02-04 13:19:22mark.dickinsonsetstatus: open -> closed
resolution: not a bug
messages: + msg210226
2014-02-04 13:10:36mdealencarsetstatus: closed -> open
title: Context.create_decimal_from_float() inconsistent precision for zeros after decimal mark -> Context setting to print Decimal with as many digits as the "prec" setting
type: behavior -> enhancement
versions: - Python 3.3
messages: + msg210225

resolution: not a bug -> (no value)
2014-02-04 11:56:50mark.dickinsonsetstatus: open -> closed
resolution: not a bug
messages: + msg210210
2014-02-04 11:42:03mdealencarsetstatus: pending -> open
resolution: not a bug -> (no value)
messages: + msg210206
2014-02-04 11:02:10mark.dickinsonsetstatus: open -> pending
resolution: not a bug
messages: + msg210199
2014-02-04 07:15:19berker.peksagsetnosy: + mark.dickinson, skrah
2014-02-03 14:28:52mdealencarcreate