classification
Title: xdrlib fails to detect overflow (struct bug?)
Type: behavior Stage: test needed
Components: Extension Modules Versions: Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mark.dickinson Nosy List: ajaksu2, lloyd, mark.dickinson
Priority: normal Keywords:

Created on 2007-11-29 17:15 by lloyd, last changed 2009-07-07 15:11 by mark.dickinson. This issue is now closed.

Files
File name Uploaded Description Edit
xdr.py lloyd, 2007-11-29 17:15
Messages (5)
msg57939 - (view) Author: Jack Lloyd (lloyd) Date: 2007-11-29 17:15
The XDR format requires integers to fit into 4 byte values. However (at
least on x86-64) xdrlib will silently accept (and truncate) values
larger than this (up to 2**64-1). Taking a (very brief) look at the
xdrlib code, it appears this is actually a problem in the struct module,
but I don't have the time (or interest) to trace through the _struct
module code.

An example on an x86-64 machine (Python 2.4.3) of encoding 2**32 (which
will not fit in an XDR field) being silently truncated:

$ ./xdr.py 
4294967296 -> 00000000 -> 0

An example of struct itself not detecting overflow:

>>> struct.pack("!I", 2**32)
'\x00\x00\x00\x00'

struct.pack will only throw an overflow error if a value >= 2**64 is
used, even if it is encoding into a field that is much smaller.
msg87709 - (view) Author: Daniel Diniz (ajaksu2) Date: 2009-05-13 21:02
On trunk, I get the described struct behavior with " DeprecationWarning:
struct integer overflow masking is deprecated". The xdr.py script gives:
18446744073709551615 -> ffffffff -> 4294967295

On py3k, both raise "struct.error: argument out of range".

Tested on Linux ia32.
msg87710 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-05-13 21:07
I'll take a look.
msg90151 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-07-05 10:15
I think there's nothing we can do about this in 2.6: there's probably code 
out there that depends on the overflow behaviour.

However, 2.6 should have been issuing DeprecationWarnings for overflow 
handling, so it would be safe to remove the overflow wrapping for 2.7.
msg90237 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-07-07 15:11
The deprecated overflow wrapping has been removed from trunk in r73891.  
(It was already removed in py3k a while ago.)

I'd really like to remove the deprecated float coercion from trunk too, 
but since the warnings weren't functioning properly in 2.6, this probably 
has to wait until 2.8.
History
Date User Action Args
2009-07-07 15:11:32mark.dickinsonsetstatus: open -> closed
resolution: fixed
messages: + msg90237
2009-07-05 10:15:43mark.dickinsonsetmessages: + msg90151
2009-05-13 21:07:47mark.dickinsonsetassignee: mark.dickinson

messages: + msg87710
nosy: + mark.dickinson
2009-05-13 21:02:14ajaksu2setversions: + Python 2.6, Python 2.7, - Python 2.4
nosy: + ajaksu2

messages: + msg87709

stage: test needed
2008-01-20 19:54:52christian.heimessetpriority: normal
2007-11-29 17:15:11lloydcreate