New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test_io large file test failure on 32 bits Android platforms #71113
Comments
test_io fails on an android emulator running an x86 system image at API level 21. ====================================================================== Traceback (most recent call last):
File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_io.py", line 537, in test_large_file_ops
self.large_file_ops(f)
File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_io.py", line 330, in large_file_ops
self.assertEqual(f.seek(self.LARGE), self.LARGE)
OverflowError: Python int too large to convert to C long ====================================================================== Traceback (most recent call last):
File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_io.py", line 537, in test_large_file_ops
self.large_file_ops(f)
File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_io.py", line 330, in large_file_ops
self.assertEqual(f.seek(self.LARGE), self.LARGE)
File "/sdcard/org.bitbucket.pyona/lib/python3.6/_pyio.py", line 1632, in seek
return os.lseek(self._fd, pos, whence)
OverflowError: Python int too large to convert to C long Ran 541 tests in 32.829s FAILED (errors=2, skipped=8) |
This looks similar to bpo-11184. |
The relevant part of the output of configure on x86: checking size of int... 4 And as a consequence in pyconfig.h: |
On android x86: >>> import resource
>>> resource.setrlimit(resource.RLIMIT_FSIZE, (2**31, 2**31))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long
>>> resource.setrlimit(resource.RLIMIT_FSIZE, (2**31-1, 2**31-1))
>>> resource.getrlimit(resource.RLIMIT_FSIZE)
(2147483647, 2147483647)
>>> The test_io tests that fail may be skipped by trying first to set this resource limit (if existing) to the max file size tested, in a spawned interpreter. |
Here is a patch that uses resource.setrlimit() to check whether large file may not be supported. |
The resource module is not always available. And resource.setrlimit() itself can fail. May be check HAVE_LARGEFILE_SUPPORT directly using sysconfig.get_config_vars()? test_mmap needs similar fix. |
Skipping test is the one way. Other way is to enable large files support on Android (like it was done in bpo-11184 for AIX). It is hard to believe that Android's libc doesn't support files larger 2GiB. Should be an option to enable this support in a program. |
Using the glibc compilation flags recommended for large file support '-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1' does not enable large file support. See Suse's 'Large File Support in Linux' document [1]. In the android bpo-64613 [2] one of the last comment said, only few days ago, "fixed in the platform. (partially in L, mostly in M, completely in N.)", L standing for Lollipop (API 21 or 22) and M for Marshmallow (API 23). So it seems these tests should be skipped until platform N is available. [1] http://users.suse.com/~aj/linux_lfs.html |
Size of off_t is also 4 when building on API 23 with or without |
This patch skips the large file support test. |
The tests are run on an ext4 file system. FWIW, the second item in https://android.googlesource.com/platform/bionic.git/#32_bit-ABI-bugs may explain why there is still no large file support on Android 32 bits platforms. |
New changeset d616304b82aa by Xavier de Gaye in branch '3.6': New changeset 878f91b4ad19 by Xavier de Gaye in branch 'default': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: