This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_maxint64 fails on 32-bit systems due to assumption that 64-bit fits into "long"
Type: Stage:
Components: Build Versions: Python 2.5
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: pitrou Nosy List: lkcl, loewis, pitrou
Priority: normal Keywords:

Created on 2009-01-17 22:00 by lkcl, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (11)
msg80052 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2009-01-17 22:00
the assumption is made that the result will fit into a PyInt.
obviously, on a 32-bit system, where SIZEOF_LONG is 4 bytes,
that ain't happening.

a complex test would be something like this:

if len <= 9: it's an int, definitely.
if len > 10: it's a long, definitely.
if len == 10, and first char is a "-", it's an int, definitely
if len == 10, and first char is 5,6,7,8,9, it's a long, definitely.
if len == 10, and first char is 0,1,2,3, it's an int, definitely.
if len == 10, and first char is 4, it _might_ be a long, but it might
also be an int, so... uh... let's assume it's a long!

and you know what?  xxxx that for a game of soldiers: just use
"if len >= 10" assume it's a long :)




diff --git a/Modules/cPickle.c b/Modules/cPickle.c
index 537276c..e56f8e5 100644
--- a/Modules/cPickle.c
+++ b/Modules/cPickle.c
@@ -3143,7 +3143,15 @@ load_int(Unpicklerobject *self)
    errno = 0;
    l = strtol(s, &endptr, 0);

-   if (errno || (*endptr != '\n') || (endptr[1] != '\0')) {
+   if (errno || (*endptr != '\n') || (endptr[1] != '\0')
+#if SIZEOF_LONG == 4
+        /* integers of 10 chars or over could be bigger than 32-bit.
+         * just keep it simple: 10 or more chars, it goes into
+         * a PyLong.
+         */
+        || (len >= 10)
+#endif
+       ) {
        /* Hm, maybe we've got something long.  Let's try reading
           it as a Python long object. */
        errno = 0;
msg80058 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-17 22:51
From the included patch, I assume you're talking about a failure of
test_pickle.

I'm not seeing any such failure on my (32-bit) system, and the code in 
Modules/cPickle.c looks correct to me.  I suspect that the problem is that 
in your setup, strtol is failing to set errno correctly on overflow.
msg80059 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-17 22:57
You might try replacing the strtol call with a call to PyOS_strtol.  
Please let us know if this works!
msg80236 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-20 09:00
Closing as invalid.
msg80243 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-01-20 11:19
Reopening at OP's request.
msg80245 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-01-20 11:22
Two comments:
- please be more specific when mentioning a problem (when reading your
bug report, I had a hard time figuring out what "test_maxint64" could be)
- I hope you're testing against trunk or at least 2.6, because the 2.5
branch will only receive security fixes
msg80247 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-20 11:31
Luke,

I closed this as invalid because it's not a Python bug:  your system's
strtol is buggy, so your bug report should go elsewhere.
msg80251 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-20 12:18
Antoine, I leave it to you to decide what to do with this.

As far as I can see, the only action that one might want to take
as a result of this issue is to replace strtol with PyOS_strol
in the codebase (in the struct module, and there's also an
occurrence in Objects/floatobject.c in the fromhex method).

I recommend against this though, for a few reasons:

1. PyOS_strtol is likely slower than strtol.

2. PyOS_strtol is itself buggy:  for example, it incorrectly
accepts strings like "- 3", with whitespace between the sign
and the unsigned part.  This is the reason that Python accepts
'- 3' as an input to int;  this is a Python bug that can't be
changed in 2.x for compatibility reasons.  It's fixed in 3.x.

3. There's nothing wrong with the existing code, and changing
it risks introducing new bugs.  It's the platform that's buggy here: 
its strtol method doesn't comply with the C89 standard (or the C99
standard, for that matter).

It's true that Python has code to work around problems with standards
compliance on some platforms, and if this were a supported platform
then I'd be okay with making the change.  As it is, though, I think this
should be reclosed (as invalid, or won't fix if that makes people feel
happier).

BTW, Luke, note that you can still add comments to an issue even after
the issue has been closed:  people subscribed to that issue (via nosy)
will still see the messages.
msg80253 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2009-01-20 12:50
hiya folks,

lots of comments here.  in no particular order:

1) thanks for reopening the bug

2) apologies for not being clearer - it's Lib/test/test_testzip.py
specifically the  TestZip64InSmallFiles case that's failing.

3) it's not yet _clear_ whether strtol is the cause of the
problem - i haven't yet got round to testing that because
i'm in the middle of a rebuild trying to get msvcr80
assemblies to work, and i'll need to back out of that and
rebuild with msvcrt or msvcrt71 first.

4) i'm not yet certain as to whether it's the mingw 3.4.5
_compiler_ that's broken (!) there's some empirical evidence
suggesting that it might well be, but again, that's not
yet determined

5) under wine, strtol goes through to msvcrt, which goes through
to ntdll, which goes through to the _unix_ system's strtol.
wine is compiled with -m32, so it _should_ still all be doing
32-bit stuff, even though i'm using a 64-bit host (debian/amd64).

on balance, i'd very much appreciate this being kept open so that i
can keep track of determining what the heck is going on - i have
about five or six different things to investigate.
msg80277 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-01-20 20:53
> i'd very much appreciate this being kept open so that i
> can keep track of determining what the heck is going on

Please understand that the purpose of the bug tracker is not to keep
your personal progress notes. A tracker item, in theory, must be
formulated such that it can immediately be responded to. If not, we may
ask for further information (closing it when it is not forthcoming), or
close it right aways as invalid.

I'm tempted to close this report immediately as completely confused and
lacking focus. Please clarify immediately

a) what action you are performing
b) what is happening
c) what you expect to happen instead
d) why you believe this is a problem with Python (and not, say, with
your target system)

Please be *EXTREMELY* cautious when using the bug tracker, as any
message you post will consume somebody else's time.
msg85842 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-04-11 08:37
Closing since I haven't seen any sign of such failures on a 64-bit build.
History
Date User Action Args
2022-04-11 14:56:44adminsetgithub: 49227
2009-04-11 08:37:27pitrousetstatus: open -> closed
resolution: works for me
messages: + msg85842
2009-01-20 20:53:07loewissetnosy: + loewis
messages: + msg80277
2009-01-20 12:50:16lkclsetmessages: + msg80253
2009-01-20 12:18:22mark.dickinsonsetnosy: - mark.dickinson
2009-01-20 12:18:05mark.dickinsonsetassignee: pitrou
messages: + msg80251
2009-01-20 11:31:40mark.dickinsonsetmessages: + msg80247
2009-01-20 11:22:15pitrousetmessages: + msg80245
2009-01-20 11:20:38pitroulinkissue5010 superseder
2009-01-20 11:19:40pitrousetstatus: closed -> open
nosy: + pitrou
resolution: not a bug -> (no value)
messages: + msg80243
2009-01-20 09:00:26mark.dickinsonsetstatus: open -> closed
resolution: not a bug
messages: + msg80236
2009-01-17 22:57:50mark.dickinsonsetmessages: + msg80059
2009-01-17 22:51:11mark.dickinsonsetnosy: + mark.dickinson
messages: + msg80058
2009-01-17 22:00:59lkclcreate