classification
Title: return-ing within code timed with timeit.timeit causes wrong return value of timeit.timeit
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Ramchandra Apte, docs@python, icedream91, jonathan.ferretti, python-dev, r.david.murray, rhettinger, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2013-07-21 09:46 by icedream91, last changed 2015-01-27 08:56 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
timeit.patch jonathan.ferretti, 2014-04-14 16:49 patch containing added note review
timeit_reject_invalid_statements.patch serhiy.storchaka, 2014-12-31 18:16 review
Messages (17)
msg193429 - (view) Author: (icedream91) Date: 2013-07-21 09:46
When I ran code below, it printed -1. The question is, the code in variable snippet, has wrong syntax, it can't be run. I think timeit should return only non-negative float type.


from timeit import timeit

snippet="""
for i in range(10):
    return -1
"""

print(timeit(snippet))


Thanks.
msg193432 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2013-07-21 09:59
Well, not sure if this is worth fixing, I think this is because timeit runs a modified version of the code using exec(), with the actual code to be timed within a function. (timeit runs something like this with exec():

def actual_code():
   #The real code
   for i in range(10): return -1
<some other code>
actual_code()
msg193451 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-07-21 14:19
I'm inclined to agree with Ramchandra.  It might be worth a doc footnote, though.
msg193473 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-07-21 23:31
Agreed this should be closed as "won't fix".
msg193483 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-07-22 02:20
Rather than closing it, we converted it to a documentation issue.  I think it is worth a footnote in the docs, since it is not obvious (without reading the source code) that a return statement will cause timeit to return an invalid result instead of raising a syntax error.
msg193487 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-07-22 02:37
Go ahead and add a footnote, but do consider that such footnotes are mostly just clutter.  It doesn't help someone at the moment there is an invalid return value -- instead it just makes it so that afterward someone can say that it is documented.
msg216108 - (view) Author: jonathan ferretti (jonathan.ferretti) * Date: 2014-04-14 16:49
Added note to timeit function briefly explaining how to avoid it the issue and the cause
msg216129 - (view) Author: Roundup Robot (python-dev) Date: 2014-04-14 17:40
New changeset 7e2708484ea5 by Andrew Kuchling in branch '3.4':
#18518: mention that including a return statement changes/breaks the behaviour
http://hg.python.org/cpython/rev/7e2708484ea5
msg216136 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-04-14 17:54
The suggestion was to make this a footnote, not a note.  Also, it should probably say that the stmt is executed inside a function, meaning that instead of being a syntax error it changes the return value of the internal timeit function.

I understand Raymond's desire not to clutter the docs, but I consider the footnote worth it, not to pre-inform the user, but to let them know that it is not a bug if they check the docs *after* things don't work right.  It may be naive of me to think that they would do so.
msg216140 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2014-04-14 18:00
I dislike footnotes and prefer to put things in the text whenever possible.  The text for this change struck as relevant enough -- it notes a property of the *stmt* parameter -- that it doesn't belong in a footnote.  (I'd be happy to just make it a paragraph instead of a highlighted note: I dislike notes too!)
msg216141 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2014-04-14 18:01
BTW, this change is also relevant to 2.7.  Previously I wouldn't have bothered to commit it to 2.7 since the branch was largely closed, but now we're talking about some corporate maintainer eventually going through and backporting fixes to the newly-extended 2.7 branch.  So should I go ahead and apply it to 2.7?
msg216198 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-04-14 20:19
OK, if you think it is worthwhile in the text, then sure.  But yeah, not as a ..note.  And yes, I think we should keep backporting relevant doc patches.  Especially since Google results still land one on the 2.7 docs...
msg233264 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-31 18:16
May be add a guard against statements which can confuse timeit? These are not only "return", but "yield", and "break" and "continue" outside of a loop. Proposed patch checks that testing code can be compiled outside of a function.
msg234235 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-18 10:01
If there are no objections I'm going to commit the patch.
msg234729 - (view) Author: Roundup Robot (python-dev) Date: 2015-01-26 10:15
New changeset e8db1cbe416b by Serhiy Storchaka in branch '2.7':
Issue #18518: timeit now rejects statements which can't be compiled outside
https://hg.python.org/cpython/rev/e8db1cbe416b

New changeset a5769fa55791 by Serhiy Storchaka in branch '3.4':
Issue #18518: timeit now rejects statements which can't be compiled outside
https://hg.python.org/cpython/rev/a5769fa55791

New changeset b0a686260b5d by Serhiy Storchaka in branch 'default':
Issue #18518: timeit now rejects statements which can't be compiled outside
https://hg.python.org/cpython/rev/b0a686260b5d
msg234732 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-26 10:54
Buildbots are unhappy. Example:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%202.7/builds/837/steps/test/logs/stdio

1 test failed:
    test_timeit

Re-running test 'test_timeit' in verbose mode
test test_timeit crashed -- <type 'exceptions.ImportError'>: No module named support
Traceback (most recent call last):
  File "./Lib/test/regrtest.py", line 901, in runtest_inner
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/test/test_timeit.py", line 8, in <module>
    from test.support import run_unittest
ImportError: No module named support
[44296 refs]
*** Error code 1
msg234733 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-26 11:13
Already fixed in 617c226da195. Needs time for buildbots to rerun tests. I didn't noticed error when backported tests because there was local file support.py in my workspace.
History
Date User Action Args
2015-01-27 08:56:11serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-01-26 11:13:11serhiy.storchakasetmessages: + msg234733
2015-01-26 10:54:57vstinnersetnosy: + vstinner
messages: + msg234732
2015-01-26 10:15:36python-devsetmessages: + msg234729
2015-01-18 10:01:45serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg234235
2014-12-31 18:16:57serhiy.storchakasetfiles: + timeit_reject_invalid_statements.patch

assignee: docs@python -> (no value)
components: + Library (Lib), - Documentation
versions: + Python 3.5, - Python 3.3
nosy: + serhiy.storchaka

messages: + msg233264
stage: patch review
2014-12-31 16:25:20akuchlingsetnosy: - akuchling
2014-04-14 20:19:23r.david.murraysetmessages: + msg216198
2014-04-14 18:01:13akuchlingsetmessages: + msg216141
2014-04-14 18:00:56akuchlingsetnosy: + akuchling
messages: + msg216140
2014-04-14 17:54:39r.david.murraysetmessages: + msg216136
2014-04-14 17:40:06python-devsetnosy: + python-dev
messages: + msg216129
2014-04-14 16:49:34jonathan.ferrettisetfiles: + timeit.patch

nosy: + jonathan.ferretti
messages: + msg216108

keywords: + patch
type: enhancement -> behavior
2013-07-22 02:37:33rhettingersetmessages: + msg193487
2013-07-22 02:20:04r.david.murraysetstatus: closed -> open
resolution: wont fix -> (no value)
messages: + msg193483
2013-07-21 23:31:48rhettingersetstatus: open -> closed

nosy: + rhettinger
messages: + msg193473

resolution: wont fix
2013-07-21 17:16:19serhiy.storchakasetversions: + Python 2.7, Python 3.4
nosy: + docs@python

assignee: docs@python
components: + Documentation, - Library (Lib)
type: behavior -> enhancement
2013-07-21 14:19:31r.david.murraysetnosy: + r.david.murray
messages: + msg193451
2013-07-21 10:01:28Ramchandra Aptesettitle: return-ing within timeit.timeit causes wrong return value of timeit.timeit -> return-ing within code timed with timeit.timeit causes wrong return value of timeit.timeit
2013-07-21 10:00:58Ramchandra Aptesettitle: timeit bug? -> return-ing within timeit.timeit causes wrong return value of timeit.timeit
2013-07-21 09:59:43Ramchandra Aptesetnosy: + Ramchandra Apte
messages: + msg193432
2013-07-21 09:46:59icedream91create