This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Large file support on Windows: sizeof(off_t) is 32 bits
Type: Stage:
Components: Windows Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: loewis, pitrou, python-dev, rosslagerwall, schmir, vstinner
Priority: normal Keywords: patch

Created on 2011-07-07 22:54 by vstinner, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
fileio_py_off_t.patch vstinner, 2011-07-07 22:58 review
Messages (5)
msg139997 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-07-07 22:54
FileIO.readall() and _parse_off_t() help of the posix module use the off_t type. This type is only 32 bits long and so don't support files bigger than 4 GB (or maybe just 2 GB?). The Py_off_t type should be used instead.
msg139998 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-07-07 22:58
fileio_py_off_t.patch: Fix for FileIO.readall().

The consequence of the integer overflow in new_buffersize() looks to be that the buffer can be too small in some cases (and so readall() can be very slow?).
msg139999 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-07-07 23:03
_parse_off_t() is used by the following functions:
- lockf
- pread, pwrite
- sendfile
- truncate, ftruncate
- posix_advice, posix_fallocate

Windows has none of these functions. _parse_off_t() may be surrounded by #ifndef MS_WINDOWS with a comment explaining that Py_off_t should be used on Windows to support large files.
msg144263 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-09-19 06:32
New changeset dc16b6ab8e71 by Ross Lagerwall in branch 'default':
Issue #12517: Silence warning on windows buildbots (from 7fd80c61ddaa).
http://hg.python.org/cpython/rev/dc16b6ab8e71
msg145749 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-10-17 18:39
changeset:   72869:32b1999410de
user:        Victor Stinner <victor.stinner@haypocalc.com>
date:        Tue Oct 11 22:45:02 2011 +0200
files:       Modules/_io/fileio.c
description:
Fix io.FileIO.readall() on Windows 64 bits

Use Py_off_t type (64 bits) instead of off_t (32 bits).
History
Date User Action Args
2022-04-11 14:57:19adminsetgithub: 56726
2011-10-17 18:39:19vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg145749
2011-09-19 06:32:07python-devsetnosy: + python-dev
messages: + msg144263
2011-07-11 12:10:16rosslagerwallsetnosy: + rosslagerwall
2011-07-08 19:27:45schmirsetnosy: + schmir
2011-07-07 23:03:02vstinnersetmessages: + msg139999
2011-07-07 22:58:10vstinnersetfiles: + fileio_py_off_t.patch
keywords: + patch
messages: + msg139998
2011-07-07 22:54:09vstinnercreate