classification
Title: Add Mako template benchmark to Python Benchmark Suite
Type: enhancement Stage: resolved
Components: Benchmarks Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, bobbyi, collinwinter, pitrou
Priority: normal Keywords: patch

Created on 2010-11-17 19:23 by bobbyi, last changed 2010-11-20 22:12 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
mako.patch bobbyi, 2010-11-17 19:22 patch to add Mako
Messages (6)
msg121378 - (view) Author: Bobby Impollonia (bobbyi) Date: 2010-11-17 19:22
The Benchmark Suite currently contains two template benchmarks (Django and Spitfire) for Python 2.x, but none that support 3.x.

The attached patch adds a benchmark using Mako (http://www.makotemplates.org/), a popular, pure-Python, performance-oriented template system that supports both 2.x and 3.x via 2to3 conversion.

Mako is added to the "py3k" benchmark group and a new group called "template" with all three template benchmarks is added (that will only work on 2.x since it uses Django and Spitfire).

I added the Mako benchmark and lib to the lists in the README file and also updated the descriptions of the pickle benchmarks there since I had missed that before. Also fixed a path in there that was referring to perf.py as unladen-benchmarks/perf.py.
msg121379 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-17 19:55
Uh, after translation I get a SystemError when trying to run the Mako bench under the current py3k SVN:

$ PYTHONPATH=lib/mako/ /home/antoine/py3k/py3k/python ./performance/bm_mako.py -n 25
Traceback (most recent call last):
  File "./performance/bm_mako.py", line 26, in <module>
    """)
  File "/home/antoine/py3k/benchmarks/3/lib/mako/mako/template.py", line 183, in __init__
    (code, module) = _compile_text(self, text, filename)
  File "/home/antoine/py3k/benchmarks/3/lib/mako/mako/template.py", line 463, in _compile_text
    strict_undefined=template.strict_undefined)
  File "/home/antoine/py3k/benchmarks/3/lib/mako/mako/codegen.py", line 51, in compile
    node)
  File "/home/antoine/py3k/benchmarks/3/lib/mako/mako/codegen.py", line 98, in __init__
    defs = self.write_toplevel()
  File "/home/antoine/py3k/benchmarks/3/lib/mako/mako/codegen.py", line 196, in write_toplevel
    main_identifiers = module_identifiers.branch(self.node)
  File "/home/antoine/py3k/benchmarks/3/lib/mako/mako/codegen.py", line 866, in branch
    return _Identifiers(node, self, **kwargs)
  File "/home/antoine/py3k/benchmarks/3/lib/mako/mako/codegen.py", line 831, in __init__
    self.topleveldefs = util.SetLikeDict(**parent.topleveldefs)
SystemError: Python/getargs.c:1398: bad argument to internal function
msg121381 - (view) Author: Bobby Impollonia (bobbyi) Date: 2010-11-17 20:41
Did you convert the project using make_perf3.sh? This benchmark is in the 2to3 group and not the 2n3 group because it won't work without conversion.

Using the attached patch, the following sequence of commands works for me to run the benchmark with python 3.1:

hg clone http://hg.python.org/benchmarks/ py2benchmarks
cd py2benchmarks/
hg import mako.patch 
mkdir ../py3benchmarks
cd ../py3benchmarks
../py2benchmarks/make_perf3.sh ../py2benchmarks
python3.1 perf.py -f -b mako python3.1 python3.1
msg121383 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-17 20:43
> Did you convert the project using make_perf3.sh?

Yes. But given the error (SystemError) I guess this is a regression in
py3k. I'll apply your patch and open a separate issue to try and make
sense of the regression.
msg121433 - (view) Author: Bobby Impollonia (bobbyi) Date: 2010-11-18 03:25
It's working okay for me using a python binary that I just built off the the py3k branch on 64-bit linux.

FWIW, in the crashing line
util.SetLikeDict(**parent.topleveldefs)
, util.SetLikeDict is a subclass of dict defined in python that just adds an extra method (union) and parent.topleveldefs is an instance of the same class.
msg121794 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-20 22:12
Pushed in 40e80cede832, thank you.
(2to3'ed Mako appears 3x slower than on 2x, but it's not obvious whether it's Python 3 itself or the constructs added by 2to3; I think running 2to3ed benchmarks is a bit pointless in itself, because it's not idiomatic code)
History
Date User Action Args
2010-11-20 22:12:27pitrousetstatus: open -> closed
resolution: fixed
messages: + msg121794

stage: resolved
2010-11-18 03:25:21bobbyisetmessages: + msg121433
2010-11-17 20:43:08pitrousetmessages: + msg121383
2010-11-17 20:41:53bobbyisetmessages: + msg121381
2010-11-17 19:55:22pitrousetassignee: collinwinter ->

messages: + msg121379
nosy: + benjamin.peterson
2010-11-17 19:23:05bobbyicreate