Message407100
Concrete example of int/int not being correctly rounded on systems using x87 instructions: on those systems, I'd expect to see 1/2731 return a result of 0.00036616623947272064 (0x1.7ff4005ffd002p-12), as a result of first rounding to 64-bit precision and then to 53-bit. The correctly-rounded result is 0.0003661662394727206 (0x1.7ff4005ffd001p-12). |
|
Date |
User |
Action |
Args |
2021-11-26 21:59:02 | mark.dickinson | set | recipients:
+ mark.dickinson, tim.peters, rhettinger, steven.daprano |
2021-11-26 21:59:02 | mark.dickinson | set | messageid: <1637963942.22.0.373827242384.issue45876@roundup.psfhosted.org> |
2021-11-26 21:59:02 | mark.dickinson | link | issue45876 messages |
2021-11-26 21:59:02 | mark.dickinson | create | |
|