classification
Title: test_mmap does not handle ValueError when no large file support
Type: behavior Stage: resolved
Components: Cross-Build, Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder: test_io large file test failure on 32 bits Android platforms
View: 26926
Assigned To: Nosy List: Alex.Willmer, pitrou, python-dev, rosslagerwall, serhiy.storchaka, twouters, xdegaye
Priority: normal Keywords: patch

Created on 2016-05-03 14:00 by xdegaye, last changed 2016-05-23 12:42 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
value-error.patch xdegaye, 2016-05-22 10:14 review
Messages (8)
msg264726 - (view) Author: Xavier de Gaye (xdegaye) * (Python committer) Date: 2016-05-03 14:00
test_mmap fails on an android emulator running an x86 system image at API level 21.

======================================================================
ERROR: test_large_filesize (test.test_mmap.LargeMmapTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 755, in test_large_filesize
    with self._make_test_file(0x17FFFFFFF, b" ") as f:
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 738, in _make_test_file
    f.seek(num_zeroes)
ValueError: cannot fit 'int' into an offset-sized integer

======================================================================
ERROR: test_large_offset (test.test_mmap.LargeMmapTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 750, in test_large_offset
    with self._make_test_file(0x14FFFFFFF, b" ") as f:
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 738, in _make_test_file
    f.seek(num_zeroes)
ValueError: cannot fit 'int' into an offset-sized integer

----------------------------------------------------------------------
Ran 35 tests in 0.134s

FAILED (errors=2, skipped=6)
test test_mmap failed
1 test failed:
    test_mmap
Total duration: 0:00:01
msg264771 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-03 21:35
This is other manifestation of issue26926.
msg266065 - (view) Author: Xavier de Gaye (xdegaye) * (Python committer) Date: 2016-05-22 10:14
f.seek(number) raises ValueError (ValueError: cannot fit 'int' into an offset-sized integer) when 'number' is greater than the size allowed by off_t. ValueError is not handled in test_mmap to detect that large file is not supported.

With this patch, test_large_filesize and test_large_offset are skipped as expected on an Android emulator.
msg266067 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-22 11:27
I'm wondering in what cases other exceptions (OSError, OverflowError) can be raised? Initial test was added in issue4681.
msg266072 - (view) Author: Xavier de Gaye (xdegaye) * (Python committer) Date: 2016-05-22 12:22
On linux with large file support, OSError is raised when the file system limit is exceeded, here with ext4:

>>> f.seek(2**44 - 2**11)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
>>> f.seek(2**44 - 2**12)
17592186040320
>>> f.write(b'A' * 2**11)
2048
>>> f.write(b'A' * 2**11)
2048
>>> f.tell()
17592186044416
>>> f.write(b'A' * 2**11)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 27] File too large


On the Android emulator (no large file support):

>>> f.seek(2**31)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: cannot fit 'int' into an offset-sized integer
>>> f.seek(2**31 - 1)
2147483647
>>> f.write(b'A')
1
>>> f.tell()
-2147483648
>>> f.flush()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 75] Value too large for defined data type


Surprisingly f.write(b'A') does not raise an exception.
msg266088 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-22 16:28
When OverflowError is raised? Shouldn't it be replaced with ValueError?
msg266122 - (view) Author: Roundup Robot (python-dev) Date: 2016-05-23 05:42
New changeset bca81ea8f898 by Serhiy Storchaka in branch '3.5':
Issue #26927: Fixed test_mmap on platforms with 32-bit off_t (like Android).
https://hg.python.org/cpython/rev/bca81ea8f898

New changeset 6147a2c99db0 by Serhiy Storchaka in branch 'default':
Issue #26927: Fixed test_mmap on platforms with 32-bit off_t (like Android).
https://hg.python.org/cpython/rev/6147a2c99db0
msg266139 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-23 12:42
> When OverflowError is raised? Shouldn't it be replaced with ValueError?

At least Python implementation of io.FileIO can raise OverflowError (from os.lseek()). Thus OverflowError should be kept.
History
Date User Action Args
2016-05-23 12:42:37serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg266139
2016-05-23 05:42:31python-devsetnosy: + python-dev
messages: + msg266122
2016-05-22 16:28:55serhiy.storchakasetmessages: + msg266088
2016-05-22 12:22:40xdegayesetmessages: + msg266072
2016-05-22 11:27:41serhiy.storchakasetnosy: + pitrou, rosslagerwall
messages: + msg266067
2016-05-22 10:14:58xdegayesetstatus: closed -> open
files: + value-error.patch
title: android: test_mmap fails -> test_mmap does not handle ValueError when no large file support
messages: + msg266065

keywords: + patch
resolution: duplicate -> (no value)
2016-05-21 07:06:39xdegayelinkissue26865 dependencies
2016-05-03 21:35:47serhiy.storchakasetstatus: open -> closed

superseder: test_io large file test failure on 32 bits Android platforms

nosy: + serhiy.storchaka
messages: + msg264771
resolution: duplicate
stage: resolved
2016-05-03 14:00:33xdegayecreate