classification
Title: Pass a namespace to timeit
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: LambertDW, belopolsky, georg.brandl, peter.otten, pitrou, python-dev, rhettinger, roippi, stefanv, steven.daprano
Priority: normal Keywords: patch

Created on 2008-04-01 12:13 by peter.otten, last changed 2014-08-23 15:28 by roippi. This issue is now closed.

Files
File name Uploaded Description Edit
diff_against_2_5_1.txt peter.otten, 2008-04-01 12:13 diff against 2.5.1
timeit_global_arg.patch roippi, 2014-08-18 21:37 review
timeit_global_arg_v2.patch roippi, 2014-08-22 03:01 review
Messages (20)
msg64805 - (view) Author: Peter Otten (peter.otten) * Date: 2008-04-01 12:13
I'd like to suggest a different approach than the one taken in rev. 
54348 to improve timeit's scripting interface: allow passing it a 
namespace. Reasons:

- It has smaller overhead for functions that take an argument:
>>> def f(a): pass
...
# trunk
>>> min(ht.Timer(lambda f=f: f(42)).repeat())
0.54068493843078613
# my patch
>>> min(mt.Timer("f(42)", ns=dict(f=f)).repeat())
0.29009604454040527

- it is more flexible. Example:
# working code, requires matplotlib
from timeit import Timer
from time import sleep

def linear(i):
    sleep(.05*i)
def quadratic(i):
    sleep(.01*i**2)

x = range(10)
y = []
for a in x:
    y.append([min(Timer("f(a)", ns=dict(f=f, a=a)).repeat(1, 1))
              for f in linear, quadratic])

from pylab import plot, show
plot(x, y)
show()

The above code works unaltered inside a function, unlike the hacks 
using "from __main__ import ...".

- the implementation is simpler and should be easy to maintain.

The provided patch is against 2.5.1. If it has a chance of being 
accepted I'm willing to jump through the necessary hoops: 
documentation, tests, etc.
msg64812 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-04-01 15:48
A more general approach would be to add both 'locals' and 'globals' to
be used by exec.  At least, I would change 'ns' to 'locals'.
msg64814 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-04-01 15:53
On the second thought, I actually wanted Timer to mimic eval without
realizing that eval uses positional rather than keywords arguments. 
'locals' is obviously a bad choice for the keyword parameter because it
masks locals() builtin.  Maybe 'local_namespace'?
msg64816 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-04-01 18:50
Generally, when I use timeit from the interpreter prompt, I use "from
__main__ import *" as the setup code string. Then I can use all
currently defined global symbols directly :)
msg64838 - (view) Author: Peter Otten (peter.otten) * Date: 2008-04-02 06:42
Alexander, I'm fine with a more specific argument name. ns was what 
the Timer already used internally.

Antoine, from __main__ import name1, ..., nameN works fine on the 
command line, but inside a function you'd have to declare the names 
you want to pass to the Timer as globals which I find a bit clumsy. 
Apart from giving a syntax warning a star-import affects the generated 
bytecode and produces the (slower) LOAD_NAME instead of LOAD_FAST.
msg64857 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-04-02 14:01
On Wed, Apr 2, 2008 at 2:42 AM, Peter Otten <report@bugs.python.org> wrote:

>  Alexander, I'm fine with a more specific argument name. ns was what
>  the Timer already used internally.
>

Maybe it should be "locals" after all.  It does not look like the
conflict with builtin locals() is an issue.  Note that this is what
__import__ uses.  I still recommend adding globals argument as well
for completeness and more accurate timings when timed code uses
globals.
msg80477 - (view) Author: David W. Lambert (LambertDW) Date: 2009-01-24 20:12
This note is simply a reminder that Antoine's 'from __main__ import *' 
solution fails in python3.  Also, resolution of this issue probably 
could incorporate Issue1397474.


>>> import timeit
>>> timeit.timeit('None','from __main__ import *')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.0/timeit.py", line 227, in timeit
    return Timer(stmt, setup, timer).timeit(number)
  File "/usr/local/lib/python3.0/timeit.py", line 135, in __init__
    code = compile(src, dummy_src_name, "exec")
  File "<timeit-src>", line 2
SyntaxError: import * only allowed at module level
msg81215 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-02-05 15:07
Georg, why did you reassign this?
msg81267 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-02-06 13:47
I'm sorry, this should have been another issue. Reassigning to you.
msg85689 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-04-07 06:47
See related discussion in issue 5441 and issue 1397474.
msg224933 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-08-06 13:35
Would still be nice to have something like this. The timeit module API is still crippled, especially now that "from __main__ import *" doesn't work in a function anymore.
msg225508 - (view) Author: Ben Roberts (roippi) * Date: 2014-08-18 21:37
Attached is a patch that adds a 'global' kwarg to the Timeit constructor, which does pretty much what it says on the tin: specifies a global namespace that exec() will use.

I originally had a 'locals' arg as well (to mirror the signature of eval/exec), but realized that the local vars I was passing to exec were not available to the inner function.  Reason: the timeit module compiles/execs a *closure*, and closed-over variables and exec() simply do not play nicely.  Possible workarounds were to munge locals() into the globals() dict, or to somehow inject the variables in the locals dict into the closure.  I found neither of these options superior to simply not including a locals argument, for reasons of Least Surprise and maintainability.

Patch includes some basic tests and documentation.  I am particularly uncomfortable with writing docs so those very likely need some polish.
msg225509 - (view) Author: Ben Roberts (roippi) * Date: 2014-08-18 22:17
Correction, the name of the argument is 'globals', not 'global'.
msg225621 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-08-21 23:21
Ben, thanks for the patch. Have you signed a contributor's agreement? You can find it at https://www.python.org/psf/contrib/contrib-form/
msg225627 - (view) Author: Ben Roberts (roippi) * Date: 2014-08-22 00:45
I did sign one right after I submitted the patch.  Takes a few days for the asterisks to propagate I guess :)
msg225633 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-08-22 01:29
Ah, good. The patch looks fine to me, except that you should add "versionchanged" tags in the documentation for the added parameter.
msg225639 - (view) Author: Ben Roberts (roippi) * Date: 2014-08-22 03:01
Ah yes.

New patch improves the docs.
msg225724 - (view) Author: Roundup Robot (python-dev) Date: 2014-08-23 03:24
New changeset e0f681f4ade3 by Antoine Pitrou in branch 'default':
Issue #2527: Add a *globals* argument to timeit functions, in order to override the globals namespace in which the timed code is executed.
http://hg.python.org/cpython/rev/e0f681f4ade3
msg225725 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-08-23 03:25
Thank you, Ben! Your patch is now pushed to the default branch.
msg225750 - (view) Author: Ben Roberts (roippi) * Date: 2014-08-23 15:28
Thanks Antoine.  Cheers :-)
History
Date User Action Args
2014-08-23 15:28:53roippisetmessages: + msg225750
2014-08-23 03:25:00pitrousetstatus: open -> closed
resolution: fixed
messages: + msg225725

stage: patch review -> resolved
2014-08-23 03:24:22python-devsetnosy: + python-dev
messages: + msg225724
2014-08-22 03:01:31roippisetfiles: + timeit_global_arg_v2.patch

messages: + msg225639
2014-08-22 01:29:55pitrousetmessages: + msg225633
stage: needs patch -> patch review
2014-08-22 00:45:44roippisetmessages: + msg225627
2014-08-21 23:21:38pitrousetmessages: + msg225621
2014-08-18 22:17:15roippisetmessages: + msg225509
2014-08-18 21:37:32roippisetfiles: + timeit_global_arg.patch
keywords: + patch
messages: + msg225508
2014-08-17 19:50:46roippisetnosy: + roippi
2014-08-06 13:35:11pitrousetstage: patch review -> needs patch
messages: + msg224933
versions: + Python 3.5, - Python 3.3
2011-11-19 14:12:50ezio.melottisetversions: + Python 3.3, - Python 3.2
2010-08-26 16:21:58BreamoreBoysetstage: patch review
versions: + Python 3.2, - Python 3.1, Python 2.7
2009-04-07 06:47:44rhettingersetassignee: rhettinger ->
messages: + msg85689
2009-04-03 14:24:03stefanvsetnosy: + stefanv
2009-02-06 13:47:48georg.brandlsetassignee: pitrou -> rhettinger
messages: + msg81267
2009-02-05 15:07:46rhettingersetmessages: + msg81215
2009-02-05 11:06:28georg.brandlsetassignee: rhettinger -> pitrou
2009-02-04 20:23:13rhettingersetpriority: normal
versions: + Python 3.1, Python 2.7, - Python 2.6
2009-01-27 05:17:21rhettingersetassignee: rhettinger
nosy: + rhettinger
2009-01-24 20:12:07LambertDWsetnosy: + LambertDW
messages: + msg80477
2009-01-23 14:32:10steven.dapranosetnosy: + steven.daprano
2008-04-02 14:01:25belopolskysetmessages: + msg64857
2008-04-02 06:42:06peter.ottensetmessages: + msg64838
2008-04-01 18:50:35pitrousetnosy: + pitrou
messages: + msg64816
2008-04-01 15:53:24belopolskysetmessages: + msg64814
2008-04-01 15:48:04belopolskysetnosy: + belopolsky
messages: + msg64812
2008-04-01 12:52:15benjamin.petersonsetnosy: + georg.brandl
2008-04-01 12:13:46peter.ottencreate