Issue7743
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2010-01-20 13:12 by mark.dickinson, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Messages (7) | |||
---|---|---|---|
msg98080 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-01-20 13:12 | |
1. Another potential crash caused by Python/dtoa.c: if the bigcomp functionality is disabled by replacing "#define STRTOD_DIGLIM 40" with "#define STRTOD_DIGLIM 4000", then the following string causes a crash: >>> s = '5254406533529552661096610603582028195612589849649138922565278497589560452182570597137658742514361936194432482059988700016338656575174473559922258529459120166686600002102838072098506622244175047522649953606315120077538558010753730576321577387528008403025962370502479105305382500086822727836607781816280407336531214924364088126680234780012085291903592543223403975751852488447885154107229587846409265285440430901153525136408849880173424692750069991045196209464308187671479664954854065777039726878381767789934729895619590000470366389383963331466851379030183764964083197053338684769252973171365139701890736933147103189912528110505014483268752328506004517760913030437151571912928276140468769502257147431182910347804663250851413437345649151934269945872064326973371182115272789687312946393533547747886024677951678751174816604738791256853675690543663283782215866825e-1180' [38199 refs] >>> float(s) cmp called with a->x[a->wds-1] == 0 I haven't yet found any examples that cause this crash without bigcomp enabled. The crash is caused by a combination of two problems. (1) there's a buggy check for the smallest denormal in the "if ((aadj = ratio(delta, bs)) <= 2.) {" block in _Py_dg_strtod: the check ignores the possibility that bc.scale is nonzero. (2) The Bigint functions don't deal well with zero inputs: in particular, d2b gives strange results, and left shifting a zero Bigint can give a non-normalized result. 2. The check: if (!(word1(&rv) & LSB)) break; in _Py_dg_strtod again doesn't take into account the possibility that bc->scale is nonzero. It's supposed to be used in exact halfway cases to determine whether the current rv is already 'even' for the purposes of round-half-to-even. But for subnormal scaled rv, this check will (wrongly) always succeed, potentially giving an incorrectly rounded result. However, it seems to be difficult to trigger this bug: the input must be a subnormal exact halfway case, which implies it must have many hundred digits. On the first pass through the strtod correction loop, the approximation is likely to be out by a few ulps, so we almost never hit the (i == 0) branch. The adjustment for the second pass then typically gives *exactly* the right result, as a consequence of the floating-point addition using round-half-to-even. |
|||
msg98081 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-01-20 13:29 | |
Okay, it's not that difficult to trigger (2). With the bigcomp functionality disabled as above: AssertionError: Incorrectly rounded str->float conversion for 11651287494059419563861790709256988151903479322938522856916519154189084656466977171489691608488398792047332126810029685763620092606534076968286334920536334924763766067178320990794927368304039797998410780646182269333271282839761794603623958163297658510063352026077076106072540390412314438457161207373275477458821194440646557259102208197382844892733860255628785183174541939743301249188486945446244053689504749943655197464973191717009938776287102040358299419343976193341216682148401588363162253931420379903449798213003874174172790742957567330246138038659650118748200625752770984217933648838167281879845022933912352785884444833681591202045229462491699354638895656152216187535257259042082360747878839946016222830869374205287663441403533948204085390898399055004119873046875e-1075: expected 0x0.0d67b36890cfcp-1022, got 0x0.0d67b36890cfbp-1022 |
|||
msg98153 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-01-22 17:06 | |
Second bug listed here fixed in r77698 in the trunk. |
|||
msg98183 - (view) | Author: Chris Withers (cjw296) * | Date: 2010-01-23 12:27 | |
Out of curiosity, is it possible to write unit tests for any/all of this? > I haven't yet found any examples that cause this crash without bigcomp > enabled. I presume you meant *with* bigcomp enabled? Forgive my lack of knowledge, but why would you disable bigcomp? |
|||
msg98185 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-01-23 12:33 | |
> Out of curiosity, is it possible to write unit tests for any/all of this? There are some tests in Lib/test/test_strtod.py, that I added around two weeks ago and have been updating since; I don't have mechanism for running tests with the altered STRTOD_DIGLIM value, though. > I presume you meant *with* bigcomp enabled? I did. Thanks! > Forgive my lack of knowledge, but why would you disable bigcomp? Just as a stress test, really: a failure that occurs with bigcomp disabled might indicate a problem that could also occur with bigcomp enabled. Or someone might want to alter the value of the STRTOD_DIGLIM threshold at some later point based on performance tests; if the code is correct then it ought to be safe to do so. |
|||
msg98186 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-01-23 12:36 | |
I forgot to mention: the other reason one might want to disable the bigcomp stuff is that it seems less reliable than the rest of the code: of the 11-12 independent bugs that I found in dtoa.c recently, around 9 of them were related to the bigcomp code. |
|||
msg98199 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-01-23 20:49 | |
First bug fixed (in the trunk) in r77713. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:56:56 | admin | set | github: 51991 |
2010-01-23 20:49:21 | mark.dickinson | set | status: open -> closed resolution: fixed messages: + msg98199 |
2010-01-23 12:36:21 | mark.dickinson | set | messages: + msg98186 |
2010-01-23 12:33:04 | mark.dickinson | set | messages: + msg98185 |
2010-01-23 12:27:17 | cjw296 | set | nosy:
+ cjw296 messages: + msg98183 |
2010-01-22 17:06:30 | mark.dickinson | set | messages: + msg98153 |
2010-01-20 13:29:24 | mark.dickinson | set | messages: + msg98081 |
2010-01-20 13:14:39 | eric.smith | set | nosy:
+ eric.smith |
2010-01-20 13:12:18 | mark.dickinson | create |