Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

return-ing within code timed with timeit.timeit causes wrong return value of timeit.timeit #62718

Closed
icedream91 mannequin opened this issue Jul 21, 2013 · 17 comments
Closed
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@icedream91
Copy link
Mannequin

icedream91 mannequin commented Jul 21, 2013

BPO 18518
Nosy @rhettinger, @vstinner, @bitdancer, @serhiy-storchaka
Files
  • timeit.patch: patch containing added note
  • timeit_reject_invalid_statements.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2015-01-27.08:56:11.103>
    created_at = <Date 2013-07-21.09:46:59.985>
    labels = ['type-bug', 'library']
    title = 'return-ing within code timed with timeit.timeit causes wrong return value of timeit.timeit'
    updated_at = <Date 2015-01-27.08:56:11.103>
    user = 'https://bugs.python.org/icedream91'

    bugs.python.org fields:

    activity = <Date 2015-01-27.08:56:11.103>
    actor = 'serhiy.storchaka'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2015-01-27.08:56:11.103>
    closer = 'serhiy.storchaka'
    components = ['Library (Lib)']
    creation = <Date 2013-07-21.09:46:59.985>
    creator = 'icedream91'
    dependencies = []
    files = ['34823', '37577']
    hgrepos = []
    issue_num = 18518
    keywords = ['patch']
    message_count = 17.0
    messages = ['193429', '193432', '193451', '193473', '193483', '193487', '216108', '216129', '216136', '216140', '216141', '216198', '233264', '234235', '234729', '234732', '234733']
    nosy_count = 9.0
    nosy_names = ['rhettinger', 'vstinner', 'r.david.murray', 'docs@python', 'python-dev', 'Ramchandra Apte', 'serhiy.storchaka', 'icedream91', 'jonathan.ferretti']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue18518'
    versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

    @icedream91
    Copy link
    Mannequin Author

    icedream91 mannequin commented Jul 21, 2013

    When I ran code below, it printed -1. The question is, the code in variable snippet, has wrong syntax, it can't be run. I think timeit should return only non-negative float type.

    from timeit import timeit
    
    snippet="""
    for i in range(10):
        return -1
    """
    
    print(timeit(snippet))

    Thanks.

    @icedream91 icedream91 mannequin added type-bug An unexpected behavior, bug, or error stdlib Python modules in the Lib dir labels Jul 21, 2013
    @RamchandraApte
    Copy link
    Mannequin

    RamchandraApte mannequin commented Jul 21, 2013

    Well, not sure if this is worth fixing, I think this is because timeit runs a modified version of the code using exec(), with the actual code to be timed within a function. (timeit runs something like this with exec():

    def actual_code():
       #The real code
       for i in range(10): return -1
    <some other code>
    actual_code()

    @RamchandraApte RamchandraApte mannequin changed the title timeit bug? return-ing within timeit.timeit causes wrong return value of timeit.timeit Jul 21, 2013
    @RamchandraApte RamchandraApte mannequin changed the title return-ing within timeit.timeit causes wrong return value of timeit.timeit return-ing within code timed with timeit.timeit causes wrong return value of timeit.timeit Jul 21, 2013
    @bitdancer
    Copy link
    Member

    I'm inclined to agree with Ramchandra. It might be worth a doc footnote, though.

    @serhiy-storchaka serhiy-storchaka added docs Documentation in the Doc dir type-feature A feature request or enhancement and removed stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jul 21, 2013
    @rhettinger
    Copy link
    Contributor

    Agreed this should be closed as "won't fix".

    @bitdancer
    Copy link
    Member

    Rather than closing it, we converted it to a documentation issue. I think it is worth a footnote in the docs, since it is not obvious (without reading the source code) that a return statement will cause timeit to return an invalid result instead of raising a syntax error.

    @bitdancer bitdancer reopened this Jul 22, 2013
    @rhettinger
    Copy link
    Contributor

    Go ahead and add a footnote, but do consider that such footnotes are mostly just clutter. It doesn't help someone at the moment there is an invalid return value -- instead it just makes it so that afterward someone can say that it is documented.

    @jonathanferretti
    Copy link
    Mannequin

    jonathanferretti mannequin commented Apr 14, 2014

    Added note to timeit function briefly explaining how to avoid it the issue and the cause

    @jonathanferretti jonathanferretti mannequin added type-bug An unexpected behavior, bug, or error and removed type-feature A feature request or enhancement labels Apr 14, 2014
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Apr 14, 2014

    New changeset 7e2708484ea5 by Andrew Kuchling in branch '3.4':
    bpo-18518: mention that including a return statement changes/breaks the behaviour
    http://hg.python.org/cpython/rev/7e2708484ea5

    @bitdancer
    Copy link
    Member

    The suggestion was to make this a footnote, not a note. Also, it should probably say that the stmt is executed inside a function, meaning that instead of being a syntax error it changes the return value of the internal timeit function.

    I understand Raymond's desire not to clutter the docs, but I consider the footnote worth it, not to pre-inform the user, but to let them know that it is not a bug if they check the docs *after* things don't work right. It may be naive of me to think that they would do so.

    @akuchling
    Copy link
    Member

    I dislike footnotes and prefer to put things in the text whenever possible. The text for this change struck as relevant enough -- it notes a property of the *stmt* parameter -- that it doesn't belong in a footnote. (I'd be happy to just make it a paragraph instead of a highlighted note: I dislike notes too!)

    @akuchling
    Copy link
    Member

    BTW, this change is also relevant to 2.7. Previously I wouldn't have bothered to commit it to 2.7 since the branch was largely closed, but now we're talking about some corporate maintainer eventually going through and backporting fixes to the newly-extended 2.7 branch. So should I go ahead and apply it to 2.7?

    @bitdancer
    Copy link
    Member

    OK, if you think it is worthwhile in the text, then sure. But yeah, not as a ..note. And yes, I think we should keep backporting relevant doc patches. Especially since Google results still land one on the 2.7 docs...

    @serhiy-storchaka
    Copy link
    Member

    May be add a guard against statements which can confuse timeit? These are not only "return", but "yield", and "break" and "continue" outside of a loop. Proposed patch checks that testing code can be compiled outside of a function.

    @serhiy-storchaka serhiy-storchaka added stdlib Python modules in the Lib dir and removed docs Documentation in the Doc dir labels Dec 31, 2014
    @serhiy-storchaka
    Copy link
    Member

    If there are no objections I'm going to commit the patch.

    @serhiy-storchaka serhiy-storchaka self-assigned this Jan 18, 2015
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jan 26, 2015

    New changeset e8db1cbe416b by Serhiy Storchaka in branch '2.7':
    Issue bpo-18518: timeit now rejects statements which can't be compiled outside
    https://hg.python.org/cpython/rev/e8db1cbe416b

    New changeset a5769fa55791 by Serhiy Storchaka in branch '3.4':
    Issue bpo-18518: timeit now rejects statements which can't be compiled outside
    https://hg.python.org/cpython/rev/a5769fa55791

    New changeset b0a686260b5d by Serhiy Storchaka in branch 'default':
    Issue bpo-18518: timeit now rejects statements which can't be compiled outside
    https://hg.python.org/cpython/rev/b0a686260b5d

    @vstinner
    Copy link
    Member

    Buildbots are unhappy. Example:

    http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%202.7/builds/837/steps/test/logs/stdio

    1 test failed:
    test_timeit

    Re-running test 'test_timeit' in verbose mode
    test test_timeit crashed -- <type 'exceptions.ImportError'>: No module named support
    Traceback (most recent call last):
      File "./Lib/test/regrtest.py", line 901, in runtest_inner
      File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/test/test_timeit.py", line 8, in <module>
        from test.support import run_unittest
    ImportError: No module named support
    [44296 refs]
    *** Error code 1

    @serhiy-storchaka
    Copy link
    Member

    Already fixed in 617c226da195. Needs time for buildbots to rerun tests. I didn't noticed error when backported tests because there was local file support.py in my workspace.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants