Message119247
(Discovered in that StackOverflow answer: http://stackoverflow.com/questions/3940518/3942509#3942509 ; check the comments too)
operator.attrgetter in its simplest form (i.e. with a single non-dotted name) needs more time to execute than an equivalent lambda expression.
Attached file so3940518.py runs a simple benchmark comparing: a list comprehension of plain attribute access; attrgetter; and lambda. I will append sample benchmark times at the end of the comment.
Browsing Modules/operator.c, I noticed that the dotted_getattr function was using PyUnicode_Check and (possibly) splitting on dots on *every* call of the attrgetter, which I thought to be most inefficient.
I changed the py3k-daily-snapshot source to make the PyUnicode_Check calls in the attrgetter_new function; also, I modified the algorithm to pre-parse the operator.attrgetter functions for possible dots in the names, in order for the dotted_getattr function to become speedier.
The only “drawback” is that now operator.attrgetter raises a TypeError on creation, not on subsequent calls of the attrgetter object; this shouldn't be a compatibility problem. However, I obviously had to update both Doc/library/operator.rst and Lib/test/test_operator.py .
I am not sure whether I should attach a zip/tar file with both the attachments (the sample benchmark and the diff); so I'll attach the diff in a further comment.
On the Ubuntu server 9.10 where I made the changes, I ran the so3940518.py sample benchmark before and after the changes.
Run before the changes (third column is seconds, less is better):
list comp 0.40999999999999925 1000000
map attrgetter 1.3899999999999997 1000000
map lambda 1.0099999999999998 1000000
Run after the changes:
list comp 0.40000000000000036 1000000
map attrgetter 0.5199999999999996 1000000
map lambda 0.96 1000000 |
|
Date |
User |
Action |
Args |
2010-10-21 00:10:21 | tzot | set | recipients:
+ tzot, docs@python |
2010-10-21 00:10:21 | tzot | set | messageid: <1287619821.22.0.706433716481.issue10160@psf.upfronthosting.co.za> |
2010-10-21 00:10:18 | tzot | link | issue10160 messages |
2010-10-21 00:10:14 | tzot | create | |
|