This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: expm1 may incorrectly raise OverflowError on underflow
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: mark.dickinson, miss-islington, miss-islington, miss-islington, steve.dower
Priority: normal Keywords: patch

Created on 2021-12-09 00:43 by steve.dower, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29997 merged steve.dower, 2021-12-09 00:51
PR 30012 merged miss-islington, 2021-12-09 18:32
PR 30012 merged miss-islington, 2021-12-09 18:32
PR 30012 merged miss-islington, 2021-12-09 18:32
PR 30013 merged miss-islington, 2021-12-09 18:32
Messages (11)
msg408059 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-12-09 00:43
If a C runtime's math functions set errno to ERANGE, we assume it is a valid underflow if fabs(result) < 1.0.

However, because expm1 includes a -1.0, it underflows towards -1.0. This fails the above check, and so if a runtime's expm1() sets ERANGE we will raise a spurious OverflowError.
msg408082 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-12-09 08:12
It's a bit cheap and nasty, but I think we could simply replace the line:

    if (fabs(x) < 1.0)

in is_error with

    if (fabs(x) < 2.0)

perhaps with an explanatory comment. All we need to do is distinguish underflow from overflow, and 2.0 is still clearly a _long_ way away from any overflow boundary.

It would be good to have a test that would trigger the behaviour, too.
msg408083 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-12-09 08:19
I presume this is also worth an upstream report? Setting ERANGE on a result that's close to -1.0 is rather questionable.
msg408137 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-12-09 16:37
I considered just switching to <2.0, but wasn't sure if I would be breaking some other unspoken behaviour there. But you're right, it's really just detecting underflow vs. overflow, so that's a much simpler way to check.

I've filed the upstream report. I suspect the errno is coming from the exp() call within the expm1() implementation, so it's underflowing towards 0.0 and then subtracting 1.
msg408139 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-12-09 16:46
I've also got no idea how to write a test for this, given that it's a very thin wrapper around a platform's C runtime library. Our existing test discovered when the library changed behaviour to start setting errno, which is probably the best we can do.
msg408145 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-12-09 17:04
> I've also got no idea how to write a test for this

Yep, that's fine. All I want is that at least one particular value that caused the spurious OverflowError is in the test suite somewhere, but it sounds as though that's already the case. I'd imagine that one of these two testcases should be enough to trigger it:

https://github.com/python/cpython/blob/44b0e76f2a80c9a78242b7542b8b1218d244af07/Lib/test/math_testcases.txt#L495-L496
msg408148 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-12-09 17:14
Lines 500-504 are the ones that trigger it. Apparently there are no tests in that file for straight exp(), but the equivalent tests for those would return 0.0 and suppress ERANGE too.
msg408151 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-12-09 18:30
> Lines 500-504 are the ones that trigger it.

Ah, right. Thanks.

> Apparently there are no tests in that file for straight exp()

Yes - that file was mostly written to give good coverage for places where we'd written our own implementations rather than simply wrapping an existing libm function, though I think we've now reverted to using the libm expm1 in all cases.
msg408152 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-12-09 18:32
New changeset 3363e1cb05d0d19ed172ea63606d8cb6268747fc by Steve Dower in branch 'main':
bpo-46018: Ensure that math.expm1 does not raise on underflow (GH-29997)
https://github.com/python/cpython/commit/3363e1cb05d0d19ed172ea63606d8cb6268747fc
msg408154 - (view) Author: miss-islington (miss-islington) Date: 2021-12-09 18:56
New changeset 5ae4265b8c8042c496e569b6dbf9ef107e1d5b31 by Miss Islington (bot) in branch '3.9':
bpo-46018: Ensure that math.expm1 does not raise on underflow (GH-29997)
https://github.com/python/cpython/commit/5ae4265b8c8042c496e569b6dbf9ef107e1d5b31
msg408157 - (view) Author: miss-islington (miss-islington) Date: 2021-12-09 19:37
New changeset ca08655b808aed2e3abeb64cb67d98a79a661dda by Miss Islington (bot) in branch '3.10':
bpo-46018: Ensure that math.expm1 does not raise on underflow (GH-29997)
https://github.com/python/cpython/commit/ca08655b808aed2e3abeb64cb67d98a79a661dda
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90176
2021-12-09 19:37:18miss-islingtonsetmessages: + msg408157
2021-12-09 19:16:56steve.dowersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-12-09 18:56:04miss-islingtonsetmessages: + msg408154
2021-12-09 18:32:15miss-islingtonsetpull_requests: + pull_request28237
2021-12-09 18:32:13miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request28236
2021-12-09 18:32:13miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request28235
2021-12-09 18:32:08miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request28234
2021-12-09 18:32:03steve.dowersetmessages: + msg408152
2021-12-09 18:30:35mark.dickinsonsetmessages: + msg408151
2021-12-09 17:14:04steve.dowersetmessages: + msg408148
2021-12-09 17:04:32mark.dickinsonsetmessages: + msg408145
2021-12-09 16:46:38steve.dowersetmessages: + msg408139
2021-12-09 16:37:56steve.dowersetmessages: + msg408137
2021-12-09 08:19:39mark.dickinsonsetmessages: + msg408083
2021-12-09 08:12:33mark.dickinsonsetmessages: + msg408082
2021-12-09 08:06:03mark.dickinsonsetnosy: + mark.dickinson
2021-12-09 00:51:00steve.dowersetkeywords: + patch
stage: patch review
pull_requests: + pull_request28220
2021-12-09 00:43:56steve.dowercreate