Message32670
Sirs:
Using python to generate a text file for transfer of data between systems, I seem to be getting inconsistent rounding results using a text formatting string
Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on win32
print "%9.4f" % (229.0 * .325,) -> 74.4250 correct
print "%9.2f" % (round(74.425, 2),) -> 74.43 correct
print "%9.2f" % (74.425,) -> 74.42 wrong
print "%9.2f" % (167.255,) -> 167.26 correct
print "%9.2f" % (167.2551,) -> 167.26 correct
print "%9.2f" % (167.2549,) -> 167.25 correct
It appears as if the string formatting code is a bit flakey. Perhaps working in binary with too small a precision?
For a bit I thought it might be "Round even up, odd down at the .005 break, but that doesn't appear to be the case.
Now that I know it's there, I can do a workaround.
Seems as if it deserves a fix or at least a document notation of the problem.
Thanks in advance,
Jim Hurlburt
Atrium Windows and Doors
Yakima, WA |
|
Date |
User |
Action |
Args |
2007-08-23 14:59:22 | admin | link | issue1778207 messages |
2007-08-23 14:59:22 | admin | create | |
|