classification
Title: range() fails with long integers
Type: behavior Stage:
Components: Interpreter Core Versions: Python 2.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, eric.smith, kszawala, mark.dickinson
Priority: normal Keywords:

Created on 2009-09-24 10:38 by kszawala, last changed 2009-12-03 12:21 by mark.dickinson. This issue is now closed.

Messages (8)
msg93063 - (view) Author: Krzysztof Szawala (kszawala) Date: 2009-09-24 10:38
range() method fails with the following error message:

"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: range() result has too many items
"

when passing a valid integer value of 9999999999.
This value is obtained from OptParse command-line option as a valid 
ingeter.

Applies to both Windows and Linux (32 and 64-bit).
msg93064 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2009-09-24 10:54
This doesn't crash the interpreter, so I'm changing it to "behavior".

The number of items in a range() must fit into a native int.

What are you doing with the range? Could you use xrange instead?
msg93066 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-09-24 11:08
I *think* range uses long internally, so 9999999999 should be okay on an
LP64 machine.  Except that of course the range() result must also fit in
memory: on a 64-bit machine range(9999999999) would need more than 300
Gb of memory.  (That's 32 bytes per entry:  24 bytes for each integer
and 8 bytes for the list pointer.)
msg93070 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-09-24 11:25
Might it make more sense for this range call to return a MemoryError
rather than an OverflowError, on 64-bit machines?
msg93071 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-09-24 11:36
Mark]
> I *think* range uses long internally

Aargh!  Sorry, Eric.  I take it back.  *xrange* uses longs internally
(and used to use ints once upon a time, IIRC), but there's a weird mix
of int and long in builtin_range that doesn't make any sense to me.  I
suspect it's historical, and may have to do both with the xrange
int->long switch and the int -> Py_ssize_t switch.
msg93095 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2009-09-24 20:53
Both range and xrange were mucked with so much just before and during py3k 
development that I am sure odd inconsistencies of what they use internally 
are oversights.
msg95928 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-12-03 12:17
I suspect the ints in builtin_range should have been changed to Py_ssize_t 
when PEP 353 was implemented.  I've fixed them in trunk in r76625;  it 
doesn't seem worth changing this in the 2.6 branch.

So on an LP64 machine with sufficient memory, range(2**31) should now 
work.
msg95929 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-12-03 12:21
Whoops. That revision number should have been r76648.
History
Date User Action Args
2009-12-03 12:21:14mark.dickinsonsetmessages: + msg95929
2009-12-03 12:17:36mark.dickinsonsetmessages: + msg95928
2009-09-24 20:53:26brett.cannonsetnosy: + brett.cannon
messages: + msg93095
2009-09-24 11:36:01mark.dickinsonsetmessages: + msg93071
2009-09-24 11:25:17mark.dickinsonsetmessages: + msg93070
2009-09-24 11:08:57mark.dickinsonsetnosy: + mark.dickinson
messages: + msg93066
2009-09-24 10:54:47eric.smithsetstatus: open -> closed

nosy: + eric.smith
messages: + msg93064

type: crash -> behavior
resolution: wont fix
2009-09-24 10:38:11kszawalacreate