classification
Title: The range for xrange() is too narrow on Windows 64-bit
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, brett.cannon, eryksun, mark.dickinson, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2016-02-24 16:40 by serhiy.storchaka, last changed 2016-02-24 21:29 by eryksun. This issue is now closed.

Files
File name Uploaded Description Edit
xrange_py_ssize_t.patch serhiy.storchaka, 2016-02-24 16:40 review
Messages (7)
msg260816 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-02-24 16:40
The xrange() object works with integers in the range of C long, not Py_ssize_t. Thus the idiomatic expression xrange(len(seq)) can fail for real sequences if sys.maxint < sys.maxsize (e.g on 64-bit Windows).

Proposed patch changes the xrange() implementation to use Py_ssize_t instead of C long.
msg260820 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-24 17:45
> (e.g on 64-bit Windows)

Platforms with sizeof(long) < sizeof(size_t), I only know one platform: Windows 64-bit.

Since this change only impacts Windows 64-bit, we must compile (to check for compilation warnings) and run tests with this patch on Windows 64-bit.

Many other similar issues has been fixed in Python 3. I'm not sure that it's worth to spend too much time on supporting the full 64-bit range on Python 2, it's almost a new feature no?

Examples:

* issue #9611: FileIO not 64-bit safe under Windows  (Python 2 fixed)
* issue #9566: Compilation warnings under x64 Windows  (not fixed in Python 2)
* issue #16367: io.FileIO.readall() is not 64-bit safe on Windows  (Python 2 fixed)
* issue #17931: PyLong_FromPid() is not correctly defined on Windows 64-bit (not fixed in Python 2?)
* issue #15792: Fix compiler options for x64 builds on Windows  (not fixed in Python 2)
* etc.
msg260822 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-02-24 17:46
I hate to say it, but I think this would count as an enhancement rather than a bugfix: there's nothing documented about the range of `xrange`.
msg260829 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-02-24 19:33
I agree that it's a new feature since xrange() still functions if you give it a value larger than `long` (right?) and there's nothing saying it has to support the full size of an index.
msg260831 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-02-24 20:03
> xrange() still functions if you give it a value larger than `long`

What do you mean? Did you mean to write range() instead of xrange()? Both range and xrange in Python 2 use a C long for the start, stop, step, and length values. With how they get used this generally isn't a problem, which is why no one previously created an issue for this.
msg260833 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-02-24 21:22
No, I meant to write xrange(). My point is there is no crasher if this change didn't occur.
msg260834 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-02-24 21:29
> No, I meant to write xrange(). My point is there is no crasher 
> if this change didn't occur.

Oh, by 'still functions' you meant that it doesn't cause an access violation that crashes the process, as opposed to raising an OverflowError that can be handled.
History
Date User Action Args
2016-02-24 21:29:50eryksunsetmessages: + msg260834
2016-02-24 21:22:07brett.cannonsetmessages: + msg260833
2016-02-24 20:03:58eryksunsetnosy: + eryksun
messages: + msg260831
2016-02-24 20:00:26serhiy.storchakasetstatus: open -> closed
resolution: rejected
stage: patch review -> resolved
2016-02-24 19:33:41brett.cannonsetnosy: + brett.cannon
messages: + msg260829
2016-02-24 17:46:51mark.dickinsonsetmessages: + msg260822
2016-02-24 17:45:49vstinnersetmessages: + msg260820
title: The range for xrange() is too narrow -> The range for xrange() is too narrow on Windows 64-bit
2016-02-24 16:40:45serhiy.storchakacreate