Message276598
Nick: Looks like it's quite a bit more work than just a dict lookup. That PyImport_ImportModuleNoBlock call (which seems odd; the implementation of NoBlock is just to wrap the blocking function; guess we don't allow non-blocking imports anymore and this is just to avoid changing all the names elsewhere?) involves a *lot* more work than just a dict lookup (it devolves to a PyImport_Import call https://hg.python.org/cpython/file/3.5/Python/import.c#l1743 , which basically does everything involved in the import process aside from actually reading/parsing the file unconditionally, because of how weird __import__ overrides can be, I guess).
While it's not a perfect comparison, compare:
>>> import _strptime # It's now cached
# Cache globals dict for fair comparison without globals() call overhead
>>> g = globals()
# Reimport (this might be *more* expensive at C layer, see notes below)
>>> %timeit -r5 import _strptime
1000000 loops, best of 5: 351 ns per loop
# Dict lookup (should be at least a bit cheaper at C layer if done equivalently, using GetAttrId to avoid temporary str)
>>> %timeit -r5 g['_strptime']
10000000 loops, best of 5: 33.1 ns per loop
# Cached reference (should be *much* cheaper at C layer)
>>> %timeit -r5 _strptime
100000000 loops, best of 5: 19.1 ns per loop
Note: I'm a little unclear on whether a Python function implemented in C has its own globals, or whether it's simulated as part of the C module initialization); if it lacks globals, then the work done for PyImport_Import looks like it roughly doubles (it has to do all sorts of work to simulate globals and the like), so that 351 ns per re-import might actually be costlier in C.
Either way, it's a >10x increase in cost to reimport compared to a dict lookup, and ~18x speedup over using a cached reference (and like I said, I think the real cost of the cheaper options would be much less in C, so the multiplier is higher). Admittedly, in tests, empty string calls to `_strptime._strptime` take around 7.4 microseconds (with realistic calls taking 8.5-13.5 microseconds), so caching is saving maybe a third of a microsecond overhead, maybe 2.5%-4.5% of the work involved in the strptime call. |
|
Date |
User |
Action |
Args |
2016-09-15 19:16:43 | josh.r | set | recipients:
+ josh.r, ncoghlan, belopolsky, palm.kevin, Roman.Evstifeev, Denny Weinberg, ammar2 |
2016-09-15 19:16:43 | josh.r | set | messageid: <1473967003.6.0.197941688651.issue27400@psf.upfronthosting.co.za> |
2016-09-15 19:16:43 | josh.r | link | issue27400 messages |
2016-09-15 19:16:43 | josh.r | create | |
|