Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_mmap does not handle ValueError when no large file support #71114

Closed
xdegaye mannequin opened this issue May 3, 2016 · 8 comments
Closed

test_mmap does not handle ValueError when no large file support #71114

xdegaye mannequin opened this issue May 3, 2016 · 8 comments
Labels
build The build process and cross-build stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@xdegaye
Copy link
Mannequin

xdegaye mannequin commented May 3, 2016

BPO 26927
Nosy @Yhg1s, @pitrou, @xdegaye, @serhiy-storchaka, @moreati
Superseder
  • bpo-26926: test_io large file test failure on 32 bits Android platforms
  • Files
  • value-error.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2016-05-23.12:42:37.783>
    created_at = <Date 2016-05-03.14:00:33.521>
    labels = ['type-bug', 'library', 'build']
    title = 'test_mmap does not handle ValueError when no large file support'
    updated_at = <Date 2016-05-23.12:42:37.781>
    user = 'https://github.com/xdegaye'

    bugs.python.org fields:

    activity = <Date 2016-05-23.12:42:37.781>
    actor = 'serhiy.storchaka'
    assignee = 'none'
    closed = True
    closed_date = <Date 2016-05-23.12:42:37.783>
    closer = 'serhiy.storchaka'
    components = ['Library (Lib)', 'Cross-Build']
    creation = <Date 2016-05-03.14:00:33.521>
    creator = 'xdegaye'
    dependencies = []
    files = ['42942']
    hgrepos = []
    issue_num = 26927
    keywords = ['patch']
    message_count = 8.0
    messages = ['264726', '264771', '266065', '266067', '266072', '266088', '266122', '266139']
    nosy_count = 7.0
    nosy_names = ['twouters', 'pitrou', 'xdegaye', 'rosslagerwall', 'python-dev', 'serhiy.storchaka', 'Alex.Willmer']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = '26926'
    type = 'behavior'
    url = 'https://bugs.python.org/issue26927'
    versions = ['Python 3.6']

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 3, 2016

    test_mmap fails on an android emulator running an x86 system image at API level 21.

    ======================================================================
    ERROR: test_large_filesize (test.test_mmap.LargeMmapTests)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 755, in test_large_filesize
        with self._make_test_file(0x17FFFFFFF, b" ") as f:
      File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 738, in _make_test_file
        f.seek(num_zeroes)
    ValueError: cannot fit 'int' into an offset-sized integer

    ======================================================================
    ERROR: test_large_offset (test.test_mmap.LargeMmapTests)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 750, in test_large_offset
        with self._make_test_file(0x14FFFFFFF, b" ") as f:
      File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 738, in _make_test_file
        f.seek(num_zeroes)
    ValueError: cannot fit 'int' into an offset-sized integer

    Ran 35 tests in 0.134s

    FAILED (errors=2, skipped=6)
    test test_mmap failed
    1 test failed:
    test_mmap
    Total duration: 0:00:01

    @xdegaye xdegaye mannequin added stdlib Python modules in the Lib dir build The build process and cross-build type-bug An unexpected behavior, bug, or error labels May 3, 2016
    @serhiy-storchaka
    Copy link
    Member

    This is other manifestation of bpo-26926.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 22, 2016

    f.seek(number) raises ValueError (ValueError: cannot fit 'int' into an offset-sized integer) when 'number' is greater than the size allowed by off_t. ValueError is not handled in test_mmap to detect that large file is not supported.

    With this patch, test_large_filesize and test_large_offset are skipped as expected on an Android emulator.

    @xdegaye xdegaye mannequin reopened this May 22, 2016
    @xdegaye xdegaye mannequin changed the title android: test_mmap fails test_mmap does not handle ValueError when no large file support May 22, 2016
    @serhiy-storchaka
    Copy link
    Member

    I'm wondering in what cases other exceptions (OSError, OverflowError) can be raised? Initial test was added in bpo-4681.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 22, 2016

    On linux with large file support, OSError is raised when the file system limit is exceeded, here with ext4:

    >>> f.seek(2**44 - 2**11)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 22] Invalid argument
    >>> f.seek(2**44 - 2**12)
    17592186040320
    >>> f.write(b'A' * 2**11)
    2048
    >>> f.write(b'A' * 2**11)
    2048
    >>> f.tell()
    17592186044416
    >>> f.write(b'A' * 2**11)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 27] File too large

    On the Android emulator (no large file support):

    >>> f.seek(2**31)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: cannot fit 'int' into an offset-sized integer
    >>> f.seek(2**31 - 1)
    2147483647
    >>> f.write(b'A')
    1
    >>> f.tell()
    -2147483648
    >>> f.flush()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 75] Value too large for defined data type

    Surprisingly f.write(b'A') does not raise an exception.

    @serhiy-storchaka
    Copy link
    Member

    When OverflowError is raised? Shouldn't it be replaced with ValueError?

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 23, 2016

    New changeset bca81ea8f898 by Serhiy Storchaka in branch '3.5':
    Issue bpo-26927: Fixed test_mmap on platforms with 32-bit off_t (like Android).
    https://hg.python.org/cpython/rev/bca81ea8f898

    New changeset 6147a2c99db0 by Serhiy Storchaka in branch 'default':
    Issue bpo-26927: Fixed test_mmap on platforms with 32-bit off_t (like Android).
    https://hg.python.org/cpython/rev/6147a2c99db0

    @serhiy-storchaka
    Copy link
    Member

    When OverflowError is raised? Shouldn't it be replaced with ValueError?

    At least Python implementation of io.FileIO can raise OverflowError (from os.lseek()). Thus OverflowError should be kept.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    build The build process and cross-build stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant