classification
Title: Bug in float.fromhex
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mark.dickinson Nosy List: mark.dickinson, miss-islington, pgimeno
Priority: normal Keywords: patch

Created on 2021-08-19 11:34 by pgimeno, last changed 2021-08-20 22:49 by terry.reedy. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27834 merged mark.dickinson, 2021-08-19 14:14
PR 27854 merged miss-islington, 2021-08-20 10:40
PR 27855 merged miss-islington, 2021-08-20 10:40
Messages (9)
msg399909 - (view) Author: Pedro Gimeno (pgimeno) Date: 2021-08-19 11:34
>>> float.fromhex('0x0.8p-1074')
0.0
>>> float.fromhex('0x.8p-1074')
5e-324

One of them is obviously wrong. It's the second one, because:
- The smallest denormal is 0x1p-1074
- Therefore, 0x0.8p-1074 is a tie for rounding purposes.
- The digit in the last place is even because the number is zero, and there is a tie, which implies rounding down.
msg399910 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-08-19 11:52
Thanks for the report! I can reproduce the issue, and agree with your analysis.
msg399913 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-08-19 12:29
The bug is in this line:

https://github.com/python/cpython/blob/3db42fc5aca320b0cac1895bc3cb731235ede794/Objects/floatobject.c#L1467

which reads:

            (half_eps == 8 && (HEX_DIGIT(key_digit+1) & 1) != 0))

In the buggy case, key_digit=0 and the HEX_DIGIT macro is trying to retrieve the value of the second-to-least significant hex digit in the coefficient, to decide whether it's odd or not. For the "0x0.8" case it retrieves the "0". For the "0x.8" case, it retrieves the "x" and tries to interpret it as a hex digit.

Even worse, if we exclude the "0x" prefix, as in `float.fromhex(".8p-1074")`, `HEX_DIGIT(1)` is accessing memory that doesn't belong to the string.
msg399960 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-08-20 10:40
New changeset 60b93d9e4922eeae25052bc15909d1f4152babde by Mark Dickinson in branch 'main':
bpo-44954: Fix wrong result in float.fromhex corner case (GH-27834)
https://github.com/python/cpython/commit/60b93d9e4922eeae25052bc15909d1f4152babde
msg399961 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-08-20 11:37
New changeset 7ef0673de48958bc3a75af5f152564bd2dffa8dd by Miss Islington (bot) in branch '3.9':
bpo-44954: Fix wrong result in float.fromhex corner case (GH-27834) (GH-27855)
https://github.com/python/cpython/commit/7ef0673de48958bc3a75af5f152564bd2dffa8dd
msg399962 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-08-20 11:40
Fixed in the main branch and in 3.9; 3.10 is very close to release, so the backport PR for 3.10 may have to wait for 3.10.1 (which wouldn't really be a problem, given that this bug has apparently lain unnoticed since Python 2.7). That's Pablo's call, of course.

@Pedro Thanks again for the report! Just out of curiosity, how did you manage to find this?
msg399983 - (view) Author: miss-islington (miss-islington) Date: 2021-08-20 17:49
New changeset 838b0e975fc2c106508eb27d19a9548533ac1e55 by Miss Islington (bot) in branch '3.10':
bpo-44954: Fix wrong result in float.fromhex corner case (GH-27834)
https://github.com/python/cpython/commit/838b0e975fc2c106508eb27d19a9548533ac1e55
msg399987 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-08-20 18:10
All fixed! Closing.
msg399991 - (view) Author: Pedro Gimeno (pgimeno) Date: 2021-08-20 20:01
> @Pedro Thanks again for the report! Just out of curiosity, how did you manage to find this?

I'm writing a C strtod implementation and I was adding corner cases to the unit testing (now published here: https://codeberg.org/pgimeno/ACSL/src/branch/master/tests/test-strtod.c). Sometimes I get confused by whether to add one or subtract one to the exponent as digits are moved, and since I have an interactive Python shell always open as a calculator, I used it to make sure that the test I was writing had the correct exponent. But the result I got was not the one I expected, and upon verification, I soon became convinced that it was a bug, so I dug a bit more (at first I didn't notice it only happened if the leading zero was left out) and finally submitted the bug.

Thanks for the quick fix, by the way.
History
Date User Action Args
2021-08-20 22:49:17terry.reedysetstatus: open -> closed
resolution: fixed
2021-08-20 20:01:50pgimenosetstatus: closed -> open
versions: - Python 3.10, Python 3.11
type: behavior ->
messages: + msg399991

resolution: fixed -> (no value)
2021-08-20 18:10:04mark.dickinsonsetstatus: open -> closed
type: behavior
messages: + msg399987

resolution: fixed
stage: patch review -> resolved
2021-08-20 17:49:04miss-islingtonsetmessages: + msg399983
2021-08-20 11:40:13mark.dickinsonsetmessages: + msg399962
2021-08-20 11:37:47mark.dickinsonsetmessages: + msg399961
2021-08-20 10:40:37miss-islingtonsetpull_requests: + pull_request26314
2021-08-20 10:40:32miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26313
2021-08-20 10:40:20mark.dickinsonsetmessages: + msg399960
2021-08-19 14:14:28mark.dickinsonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26298
2021-08-19 12:29:06mark.dickinsonsetmessages: + msg399913
2021-08-19 11:52:18mark.dickinsonsetversions: + Python 3.10, Python 3.11
nosy: + mark.dickinson

messages: + msg399910

assignee: mark.dickinson
2021-08-19 11:34:02pgimenocreate