classification
Title: decimal.to_eng_string() does not implement engineering notation in all cases.
Type: enhancement Stage:
Components: Extension Modules Versions: Python 3.6
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Keith.Brafford, eric.smith, ezio.melotti, facundobatista, mark.dickinson, rhettinger, serge.stroobandt, skrah, tim.peters, ztane
Priority: normal Keywords:

Created on 2016-01-27 19:15 by serge.stroobandt, last changed 2016-12-06 00:01 by serge.stroobandt. This issue is now closed.

Files
File name Uploaded Description Edit
engineering_notation.pdf serge.stroobandt, 2016-01-27 19:15 Proper definition of engineering notation
Messages (18)
msg259047 - (view) Author: Serge Stroobandt (serge.stroobandt) Date: 2016-01-27 19:15
In https://docs.python.org/2/library/string.html#formatstrings the proprietary (IBM) specifcation "Decimal Arithmetic Specification" http://www.gobosoft.com/eiffel/gobo/math/decimal/daconvs.html is incorrectly being heralded as "the" specifiaction for engineering notation.

However, upon reading this IBM specifation carefully, one will note that the specifaction itself actually admits not applying the engineering notation in the case of infinite numbers.

An emphasized version of the exact quote accompanied with a discussion can be found here: http://stackoverflow.com/a/17974598/2192488

Correct behaviour for decimal.to_eng_string() would be to equally employ engineering notation in the case of infinite numbers.

I suggest renaming the current behaviour to decimal.to_ibm_string().

References:
http://www.augustatech.edu/math/molik/notation.pdf
https://en.wikipedia.org/wiki/Engineering_notation
https://en.wikipedia.org/wiki/General_Conference_on_Weights_and_Measures
http://www.bipm.org/en/CGPM/db/11/11/

PS: I am a MSc in Electronic Engineering.
msg259054 - (view) Author: Serge Stroobandt (serge.stroobandt) Date: 2016-01-27 20:16
An emphasized version of the exact quote is here now:
http://stackoverflow.com/a/35045233/2192488
msg259058 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-01-27 20:33
The decimal module strives to exactly follow the spec even when our sensibilities suggest otherwise. Perhaps, we can add a note to the current docs describing the situation.  The API itself is a settled issue, that ship sailed a very long time ago (the problem with a standard library is that it becomes standard that people rely on and is hard to change after the fact without causing issues for users).
msg259059 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-01-27 20:41
Agreed. And, since any API change would just be a 3.6+ change, this would increase the difficulty of moving between 2.7 and 3.x. Which is not something we want.
msg259061 - (view) Author: Serge Stroobandt (serge.stroobandt) Date: 2016-01-27 20:46
@rhettinger
I completely agree with not creating a backward incompatibility at this point in time.

The real issue is that decimal.to_eng_string() was written to a (unfortunately chosen) proprietary specification which does not entirely correspond to the engineering notation.

A quick web search shows that a lot of people are in search of a *true* engineering notation implementation. In the phylosophy of "batteries included" it is a pity this useful and very common notation is currently missing in Python.

I would therefore suggest adding a decimal.to_true_eng_string() with the true engineering notation.

Hence, this bug could be reclassified as asuggestion for enhancement.
msg259062 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016-01-27 21:01
The spec was the only reasonable choice at the time decimal.py was
written.  Incidentally, it is basically IEEE-754-2008 with arbitrary
precision extensions; this isn't surprising since Mike Cowlishaw was
on the IEEE committee and wrote the spec at the same time.

There are very few decNumber-specific remainders in the spec -- this
may be one of those (though I did not bother to look up if the IEEE
standard specifies formatting at all).
msg259107 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-01-28 08:07
I also agree that we shouldn't change the current code. As Raymond says, it may be worth a doc change.

Serge: I was confused by your initial report. If I understand the StackOverflow question correctly, this isn't about the output for *infinite* numbers (e.g., `Decimal('inf')` and `Decimal('-inf')`), and I'm not sure what that would mean. Rather, it's about the output for small finite numbers, where an exponent wouldn't be used in the normal scientific notation. So some people would (understandably) rather see:

>>> Decimal('123456').to_eng_string()
'123.456e3'
>>> Decimal('0.02').to_eng_string()
'20e-3'

than the current

>>> Decimal('123456').to_eng_string()
'123456'
>>> Decimal('0.02').to_eng_string()
'0.02'

for example. Is that what you meant? 

Stefan: IEEE 754 does cover formatting (in section 5.12, "Details of conversion between floating-point data and external character sequences"), but has nothing to say about engineering formats.
msg259118 - (view) Author: Serge Stroobandt (serge.stroobandt) Date: 2016-01-28 10:10
Mark: Don't shoot the messenger!
I literally quoted the implemented proprietary specification.
However, I do agree that the term "numbers (or bases) with an infinte decimal representation" would be more appropriate in this context.

Also, improving documentation is good, but having a new function with the desired *true* engineering notation would be even better! Admittedly, this was my ultimate objective for filing this enhancement bug.

Thanks for commenting on StackExchange.
msg259271 - (view) Author: Serge Stroobandt (serge.stroobandt) Date: 2016-01-30 22:27
Related issue:
https://bugs.python.org/issue8060
msg259772 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-02-07 09:27
I'm disinclined to make a new method and instead prefer to go the route of having a formatting option.  For the most part, we want the methods to be limited to those in the spec (the API is already huge).  The other reason is that output formatting options are what __format__() method was intended to address.

If you want to go further, it would be reasonable to bring https://bugs.python.org/issue8060 back to life.  The final entry in that tracker item recommended moving the discussion to python ideas or into a PEP (like was done for the thousands separator format option).  There are many viewpoints to consider before jumping to codify one particular approach into the standard library.
msg272415 - (view) Author: Antti Haapala (ztane) * Date: 2016-08-11 07:55
Indeed engineering notation is now utterly broken, the engineering notation is not printed for pretty much _any *engineering* numbers at all_ in 3.6. Engineering numbers mean numbers that could be met in an *engineering* context, not cosmological!
msg272417 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016-08-11 08:03
@Antti Please think before you write and stop making unfounded allegations.
msg272428 - (view) Author: Antti Haapala (ztane) * Date: 2016-08-11 09:17
Ok, after reading the "spec" it seems that the engineering exponent is indeed printed for positive exponents *if* the precision of the number is less than the digits of the exponent, which I didn't realize that I should be testing. 

However the *precision* of decimals is meaningless anyhow. Add a very precisely measured '0e0' to any number and the sum also has exponent of 0, and is thus never displayed in exponential notation.
msg272431 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016-08-11 09:41
On Thu, Aug 11, 2016 at 09:17:10AM +0000, Antti Haapala wrote:
> However the *precision* of decimals is meaningless anyhow. Add a very precisely measured '0e0' to any number and the sum also has exponent of 0, and is thus never displayed in exponential notation.

It is not meaningless and actually one of the most important features of decimal:

>>> x = Decimal("3.6")
>>> y = Decimal("0.0000000000000000000000") # number "measured" with ridiculous precision
>>> x.to_eng_string()
'3.6'
>>> (x + y).to_eng_string()
'3.6000000000000000000000'

>>> x = Decimal("3.6")
>>> y = Decimal("0e-7") # perhaps more realistic
>>> (x + y).to_eng_string()
'3.6000000'

If you have confidence in your measurement, you have to let decimal know
by actually spelling it out.
msg272562 - (view) Author: Serge Stroobandt (serge.stroobandt) Date: 2016-08-12 20:38
What most engineers would like to see implemented in Python is a new engineering notation identical to the one implemented in the omnipresent HP calculators.

Quoting from the HP-15C Owner's Handbook:
"- In engineering notation, the first significant digit is always present in the display. The number you key in after f ENG specifies the number of additional digits to which you want to round the display.
- Engineering notation shows all exponents in multiples of three."

Source + examples, see page 59: http://www.hp.com/ctg/Manual/c03030589.pdf

Most of the time, engineers are not after high precision. Ball park figures are good enough in a world where everything is built to a specified tolerance. For example, most electronic resistors feature 5% tolerance. Safety factors take care of the rest and assure a building will not collapse.

This should not be that difficult to implement?
I promise, every six months an engineer will stop by here asking for this. Instead of nagging, this could already have been implemented one way or the other. The large demand for this feature really warrants it. Thanks!
msg272563 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016-08-12 20:56
On Fri, Aug 12, 2016 at 08:38:52PM +0000, Serge Stroobandt wrote:
> This should not be that difficult to implement?
> I promise, every six months an engineer will stop by here asking for this. Instead of nagging, this could already have been implemented one way or the other. The large demand for this feature really warrants it. Thanks!

To whom should I send the invoice?
msg272565 - (view) Author: Keith Brafford (Keith.Brafford) Date: 2016-08-12 21:15
Serge, I wrote this awhile back, before I learned you aren't supposed to subclass built-in types.  Is this the type of effect you're looking for?

https://gist.github.com/kbrafford/da39e06d18b6df2a07777eecb4493699

Here's an example using it:
https://gist.github.com/kbrafford/e0115e796890fcefb4f0c35248bd05f1
msg282487 - (view) Author: Serge Stroobandt (serge.stroobandt) Date: 2016-12-06 00:01
Dear Keith, that is exactly how it should be! (I cross-checked with a HP calculator to make sure.)
History
Date User Action Args
2016-12-06 00:01:37serge.stroobandtsetmessages: + msg282487
2016-08-12 21:15:00Keith.Braffordsetmessages: + msg272565
2016-08-12 20:56:04skrahsetmessages: + msg272563
2016-08-12 20:38:51serge.stroobandtsetmessages: + msg272562
2016-08-11 09:41:10skrahsetmessages: + msg272431
2016-08-11 09:17:10ztanesetmessages: + msg272428
2016-08-11 08:03:53skrahsetmessages: + msg272417
2016-08-11 07:55:20ztanesetnosy: + ztane
messages: + msg272415
2016-02-07 09:27:54rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg259772
2016-01-30 22:27:53serge.stroobandtsetmessages: + msg259271
2016-01-28 10:10:56serge.stroobandtsettype: behavior -> enhancement
messages: + msg259118
2016-01-28 08:07:25mark.dickinsonsetmessages: + msg259107
2016-01-27 21:01:33skrahsetmessages: + msg259062
2016-01-27 20:46:33serge.stroobandtsetmessages: + msg259061
2016-01-27 20:41:30eric.smithsetmessages: + msg259059
2016-01-27 20:33:17rhettingersetassignee: rhettinger

messages: + msg259058
nosy: + facundobatista, rhettinger, tim.peters, skrah, mark.dickinson
2016-01-27 20:16:28serge.stroobandtsetmessages: + msg259054
2016-01-27 19:50:13SilentGhostsetversions: + Python 3.6, - Python 2.7
2016-01-27 19:15:13serge.stroobandtcreate