Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rangeiter_new fails when creating a range of step 0 #72562

Closed
orenmn mannequin opened this issue Oct 6, 2016 · 14 comments
Closed

rangeiter_new fails when creating a range of step 0 #72562

orenmn mannequin opened this issue Oct 6, 2016 · 14 comments
Assignees
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@orenmn
Copy link
Mannequin

orenmn mannequin commented Oct 6, 2016

BPO 28376
Nosy @methane, @serhiy-storchaka, @Vgr255, @orenmn
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • CPythonTestOutput.txt: test output of CPython without my patches (tested on my PC)
  • patchedCPythonTestOutput_ver1.txt: test output of CPython with my patches (tested on my PC) - ver1
  • issue28376_ver1.diff: proposed patches diff file - ver1
  • issue28376_CPython35_ver2.diff: proposed patches diff file for 3.5 - ver2
  • issue28376_CPython36_ver2.diff: proposed patches diff file for 3.6 - ver2
  • issue28376_CPython37_ver2.diff: proposed patches diff file for 3.7 - ver2
  • patchedCPython35TestOutput_ver2.txt: test output of CPython3.5 with my patches (tested on my PC) - ver2
  • patchedCPython36TestOutput_ver2.txt: test output of CPython3.6 with my patches (tested on my PC) - ver2
  • patchedCPython37TestOutput_ver2.txt: test output of CPython3.7 with my patches (tested on my PC) - ver2
  • issue28376_CPython36_ver3.diff: proposed patches diff file for 3.6 - ver3
  • issue28376_CPython37_ver3.diff: proposed patches diff file for 3.7 - ver3
  • patchedCPython36TestOutput_ver3.txt: test output of CPython3.6 with my patches (tested on my PC) - ver3
  • patchedCPython37TestOutput_ver3.txt: test output of CPython3.7 with my patches (tested on my PC) - ver3
  • issue28376_CPython36_ver4.diff: proposed patches diff file for 3.6 - ver4
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2016-10-08.19:09:37.726>
    created_at = <Date 2016-10-06.12:00:27.898>
    labels = ['interpreter-core', 'type-bug', '3.7']
    title = 'rangeiter_new fails when creating a range of step 0'
    updated_at = <Date 2017-03-31.16:36:37.971>
    user = 'https://github.com/orenmn'

    bugs.python.org fields:

    activity = <Date 2017-03-31.16:36:37.971>
    actor = 'dstufft'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2016-10-08.19:09:37.726>
    closer = 'serhiy.storchaka'
    components = ['Interpreter Core']
    creation = <Date 2016-10-06.12:00:27.898>
    creator = 'Oren Milman'
    dependencies = []
    files = ['44984', '44985', '44986', '44995', '44996', '44997', '44998', '44999', '45000', '45018', '45019', '45020', '45021', '45022']
    hgrepos = []
    issue_num = 28376
    keywords = ['patch']
    message_count = 14.0
    messages = ['278187', '278188', '278189', '278197', '278232', '278234', '278237', '278238', '278307', '278309', '278313', '278315', '278316', '278352']
    nosy_count = 5.0
    nosy_names = ['methane', 'python-dev', 'serhiy.storchaka', 'abarry', 'Oren Milman']
    pr_nums = ['552']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue28376'
    versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

    @orenmn
    Copy link
    Mannequin Author

    orenmn mannequin commented Oct 6, 2016

    ------------ current state ------------

    An assertion in Objects/rangeobject.c might fail:
        >>> type(iter(range(0)))
        <class 'range_iterator'>
        >>> type(iter(range(0)))(1, 1, 0)
        Assertion failed: step != 0, file ..\Objects\rangeobject.c, line 895

    This is caused by the lack of a check whether 'step' is zero, during the creation of a range_iterator object, in rangeiter_new.

    Note that during the creation of a range object, the function range_new does check that, by calling validate_step, which leads to the following behavior:
        >>> range(1, 1, 0)
        Traceback (most recent call last):
          File "<stdin>", line 1, in <module>
        ValueError: range() arg 3 must not be zero
        >>>

    ------------ proposed changes ------------
    1. In Objects/rangeobject.c in rangeiter_new:
    - in case 'step' is zero, raise a ValueError
    - in error messages, replace 'rangeiter' with 'range_iterator', as the latter is the name of the type in Python code

    2. In [Lib/test/test_range.py](https://github.com/python/cpython/blob/main/Lib/test/test_range.py), add tests for calling the range_iterator type (i.e. creating a range_iterator object)
    

    ------------ diff ------------
    The proposed patches diff file is attached.

    ------------ tests ------------
    I ran 'python_d.exe -m test -j3' (on my 64-bit Windows 10) with and without the patches, and got quite the same output. (That also means my new tests in test_range passed.)
    The outputs of both runs are attached.

    @orenmn orenmn mannequin added type-crash A hard crash of the interpreter, possibly with a core dump 3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Oct 6, 2016
    @Vgr255
    Copy link
    Mannequin

    Vgr255 mannequin commented Oct 6, 2016

    I'm not able to trigger an assertion error when running the latest trunk in debug mode. I get a "valid" range_iterator object though, and using reduce gives me direct access to range(0, 0, 0) (which is completely worthless).

    Error or not, this should be fixed, and your patch LGTM. Thanks!

    (I changed the type to 'behaviour' since I can't reproduce the assertion, but it doesn't make much of a difference)

    @Vgr255 Vgr255 mannequin changed the title assertion failure in rangeobject.c rangeiter_new fails when creating a range of step 0 Oct 6, 2016
    @Vgr255 Vgr255 mannequin added type-bug An unexpected behavior, bug, or error and removed type-crash A hard crash of the interpreter, possibly with a core dump labels Oct 6, 2016
    @methane
    Copy link
    Member

    methane commented Oct 6, 2016

    patch is LGTM.

    But there is one hidden inconsistency:

    >>> r = range(2**100)
    >>> type(iter(r))
    <class 'longrange_iterator'>
    >>> type(iter(r))(1, 1, 0)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: cannot create 'longrange_iterator' instances

    Should we have same tp_new method for longrange_iterator?

    @serhiy-storchaka
    Copy link
    Member

    Good point Naoki. I think we can remove tp_new method from range_iterator in 3.7. Seems it is never used.

    The patch LGTM for 3.5-3.6, but the test should be marked as CPython implementation detail (@support.cpython_only).

    @orenmn
    Copy link
    Mannequin Author

    orenmn mannequin commented Oct 7, 2016

    The diff files for 3.5 and 3.6 are attached (I only added @test.support.cpython_only, as you suggested).

    The diff file for 3.7 is also attached. It contains:
    - removing rangeiter_new
    - tests to verify one cannot create instances of range_iterator or longrange_iterator (in CPython)
    - added 'Iterator.register(longrange_iterator)' in Lib/_collections_abc.py

    The output of 'python_d.exe -m test -j3' for each version is also attached (they were all successful).

    P.S. If we remove the ability to create instances of range_iterator in 3.7, shouldn't we raise a DeprecationWarning in rangeiter_new in 3.5 and 3.6?

    @serhiy-storchaka
    Copy link
    Member

    Raising a DeprecationWarning in 3.6 doesn't hurt.

    @serhiy-storchaka
    Copy link
    Member

    rangeiter_new() was added in r55167.

    @serhiy-storchaka
    Copy link
    Member

    And merged in a6eb6acfe04a.

    @orenmn
    Copy link
    Mannequin Author

    orenmn mannequin commented Oct 8, 2016

    The diff files for 3.6 and 3.7 are attached (named '*ver3.diff').
    Changes:
    - in 3.6, added:
    * raising a DeprecationWarning in rangeiter_new
    * a test to verify the DeprecationWarning is raised
    - in 3.7:
    * changed the tests so they would only verify a TypeError is raised when calling either range_iterator or longrange_iterator
    * removed the test.support.cpython_only decorator of the test

    The output of 'python_d.exe -m test -j3' for each version is also attached (they were both successful).

    @serhiy-storchaka
    Copy link
    Member

    Thanks Oren. Patches for 3.5 and 3.7 LGTM, the patch for 3.6 should be fixed. Tests should pass with -Wa and -We.

    No need to attach test output. Just check that your patch doesn't add a regression.

    @orenmn
    Copy link
    Mannequin Author

    orenmn mannequin commented Oct 8, 2016

    Alright, just added 'with test.support.check_warnings(('', DeprecationWarning)):'.
    *ver4.diff is attached.
    I ran the tests again (this time using 'python_d.exe -We -m test -j3'), and they were successful :)

    That was totally my bad, of course, but anyway, ISTM that https://docs.python.org/devguide/index.html should say './python -We -m test -j3' in section 4 (Run the tests).

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 8, 2016

    New changeset ee049edc3fff by Serhiy Storchaka in branch '3.5':
    Issue bpo-28376: Fixed typos.
    https://hg.python.org/cpython/rev/ee049edc3fff

    New changeset e486f3d30e0e by Serhiy Storchaka in branch '3.5':
    Issue bpo-28376: The constructor of range_iterator now checks that step is not 0.
    https://hg.python.org/cpython/rev/e486f3d30e0e

    New changeset 06f065a59751 by Serhiy Storchaka in branch '3.6':
    Issue bpo-28376: Creating instances of range_iterator by calling range_iterator
    https://hg.python.org/cpython/rev/06f065a59751

    New changeset e099583400f3 by Serhiy Storchaka in branch 'default':
    Issue bpo-28376: Creating instances of range_iterator by calling range_iterator
    https://hg.python.org/cpython/rev/e099583400f3

    New changeset ce4af7593e45 by Serhiy Storchaka in branch '3.5':
    Issue bpo-28376: The type of long range iterator is now registered as Iterator.
    https://hg.python.org/cpython/rev/ce4af7593e45

    @serhiy-storchaka
    Copy link
    Member

    Thank you for your contribution Oren.

    @orenmn
    Copy link
    Mannequin Author

    orenmn mannequin commented Oct 9, 2016

    Thanks for the reviews and patience :)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants