Message272914
Just to share my little experience with rounding numbers.
Last years, I worked on a API to convert timestamps between float and integers, the private "PyTime C API":
https://haypo.github.io/pytime.html
At the beginning, I used various floatting point numbers which looks fine in decimal. But quickly, I got rounding issues on various buildbots. After many years fighting against compilers and trying to write a complete test suite, I decided to only use numbers which can be stored exactly in IEEE 754:
if use_float:
# numbers with an exact representation in IEEE 754 (base 2)
for pow2 in (3, 7, 10, 15):
ns = 2.0 ** (-pow2)
ns_timestamps.extend((-ns, ns))
If you are curious, look at CPyTimeTestCase, TestCPyTime and TestOldPyTime classes in Lib/test/test_time.py.
At the end, I decided to reimplement each conversion function in pure Python using decimal.Decimal and compare the result with the C implementation. It makes the unit tests shorter and simpler. |
|
Date |
User |
Action |
Args |
2016-08-17 08:56:42 | vstinner | set | recipients:
+ vstinner, tim.peters, rhettinger, mark.dickinson, ned.deily, steven.daprano, martin.panter |
2016-08-17 08:56:42 | vstinner | set | messageid: <1471424202.01.0.281682881857.issue27761@psf.upfronthosting.co.za> |
2016-08-17 08:56:41 | vstinner | link | issue27761 messages |
2016-08-17 08:56:41 | vstinner | create | |
|