classification
Title: chameleon benchmark fails on 3.4
Type: Stage: resolved
Components: Benchmarks Versions: 3rd party
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: georg.brandl Nosy List: brett.cannon, georg.brandl, pitrou, python-dev, scoder
Priority: normal Keywords:

Created on 2013-09-23 19:42 by pitrou, last changed 2013-10-15 18:24 by georg.brandl. This issue is now closed.

Messages (9)
msg198330 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-09-23 19:42
Not sure what to do about this, but it seems the bundled Chameleon lib doesn't work on 3.4:

RuntimeError: Benchmark died: Traceback (most recent call last):
  File "./performance/bm_chameleon.py", line 38, in <module>
    util.run_benchmark(options, options.num_runs, main)
  File "/home/antoine/cpython/benchmarks/performance/util.py", line 30, in run_benchmark
    data = bench_func(num_runs, *args)
  File "./performance/bm_chameleon.py", line 19, in main
    tmpl = PageTemplate(BIGTABLE_ZPT)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/zpt/template.py", line 163, in __init__
    super(PageTemplate, self).__init__(body, **config)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/template.py", line 117, in __init__
    self.write(body)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/template.py", line 208, in write
    self.cook(body)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/template.py", line 147, in cook
    program = self._cook(body, digest, names)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/template.py", line 218, in _cook
    source = self._make(body, builtins)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/template.py", line 251, in _make
    return self._compile(module, builtins)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/template.py", line 245, in _compile
    compiler = Compiler(self.engine, program, builtins, strict=self.strict)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/compiler.py", line 886, in __init__
    generator = TemplateCodeGenerator(module)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 107, in __init__
    super(TemplateCodeGenerator, self).__init__(tree)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 172, in __init__
    self.visit(tree)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 232, in visit
    ret = visitor(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 110, in visit_Module
    super(TemplateCodeGenerator, self).visit_Module(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 239, in visit_Module
    self.visit(n)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 232, in visit
    ret = visitor(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 290, in visit_FunctionDef
    self.visit(statement)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 232, in visit
    ret = visitor(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 290, in visit_FunctionDef
    self.visit(statement)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 232, in visit
    ret = visitor(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 371, in visit_For
    self.visit(statement)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 232, in visit
    ret = visitor(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 371, in visit_For
    self.visit(statement)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 232, in visit
    ret = visitor(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 458, in visit_Try
    self.visit(statement)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 232, in visit
    ret = visitor(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 403, in visit_If
    self.visit(node.test)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 232, in visit
    ret = visitor(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 621, in _f
    f(self, node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 763, in visit_Compare
    self.visit(comparator)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/codegen.py", line 192, in visit
    super(TemplateCodeGenerator, self).visit(node)
  File "/home/antoine/cpython/benchmarks/lib3/Chameleon-2.9.2/src/chameleon/astutil.py", line 231, in visit
    node.__class__.__name__, repr(node)))
Exception: No handler for ``NameConstant`` (<_ast.NameConstant object at 0x7f4a6a733dd8>).
msg198335 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-09-23 20:11
I guess either we introduce compatible version ranges or drop Chameleon from 3.x compatibility and then add a new version of Chameleon if/when they update for Python 3.4 support.
msg198346 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-09-23 23:05
I guess version ranges could be feasible. Perhaps with benchmark-skipping decorators?
msg198354 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-09-24 12:07
Decorator sounds reasonable.
msg198355 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-09-24 12:07
Actually, one perk of the ranges is 2n3 can be auto-calculated.
msg199946 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-10-14 19:31
New changeset e40042a7788c by Georg Brandl in branch 'default':
Closes #19079: add VersionRange decorator to benchmark suite to mark compatible Python versions.
http://hg.python.org/benchmarks/rev/e40042a7788c
msg199975 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2013-10-15 04:59
This commit conflicts with the patch for issue #19108 and breaks benchmarking in Py3 (at least for me).

Traceback (most recent call last):
  File "perf.py", line 2538, in <module>
    main(sys.argv[1:])
  File "perf.py", line 2481, in main
    base_cmd_prefix, changed_cmd_prefix)
  File "perf.py", line 2318, in FilterBenchmarks
    if not minver <= basever <= maxver:
TypeError: unorderable types: str() <= bytes()

It would be better to undo the parts of this change that figure out the interpreter version, then apply the other patch, and then reimplement the filtering based on it.
msg200000 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-10-15 14:57
I'm seeing the same error as Stefan.
msg200009 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-10-15 18:24
Ah yes, I somehow missed running perf.py itself with Python 3.

Should be fixed now; adapting the #19108 patch should be easy enough.
History
Date User Action Args
2013-10-15 18:24:21georg.brandlsetstatus: open -> closed
resolution: fixed
2013-10-15 18:24:17georg.brandlsetmessages: + msg200009
2013-10-15 14:57:12brett.cannonsetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg200000
2013-10-15 06:14:41pitrousetassignee: georg.brandl

nosy: + georg.brandl
2013-10-15 04:59:54scodersetnosy: + scoder
messages: + msg199975
2013-10-14 19:31:37python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg199946

resolution: fixed
stage: resolved
2013-09-24 12:07:32brett.cannonsetmessages: + msg198355
2013-09-24 12:07:07brett.cannonsetmessages: + msg198354
2013-09-23 23:05:10pitrousetmessages: + msg198346
2013-09-23 20:11:36brett.cannonsetmessages: + msg198335
2013-09-23 19:42:22pitroucreate