After the latest changes I don't understand how I'm supposed to run translated Python 3 benchmarks:

$ ./perf.py -fb mako_v2 ../opt/python ../x32opt/python 
Running mako_v2...
INFO:root:Running ../x32opt/python ./performance/bm_mako_v2.py -n 50
Traceback (most recent call last):
  File "./perf.py", line 2460, in <module>
    main(sys.argv[1:])
  File "./perf.py", line 2416, in main
    options)))
  File "./perf.py", line 1528, in BM_mako_v2
    return SimpleBenchmark(MeasureMakoV2, *args, **kwargs)
  File "./perf.py", line 714, in SimpleBenchmark
    *args, **kwargs)
  File "./perf.py", line 1524, in MeasureMakoV2
    iteration_scaling=10)
  File "./perf.py", line 1113, in MeasureGeneric
    inherit_env=options.inherit_env)
  File "./perf.py", line 1071, in CallAndCaptureOutput
    raise RuntimeError("Benchmark died: " + stderr.decode('latin1'))
RuntimeError: Benchmark died: Traceback (most recent call last):
  File "./performance/bm_mako_v2.py", line 26, in <module>
    from mako.template import Template
  File "/home/antoine/cpython/benchmarks/lib/Mako-0.7.3/mako/template.py", line 622
    exec code in module.__dict__, module.__dict__
            ^
SyntaxError: invalid syntax

``python3 perf.py -fb mako_v2 ../opt/python ../x32opt/python`` obviously assuming you are specifying Python 3 interpreters. =)

Feel free to tweak any docs you think should be touched up to make that more obvious.

Well... It wasn't really obvious to me :-)
Why shouldn't the child interpreter decide which lib to use, though? "perf.py" is just an executable script (and its shebang specifies python, not python3).

Originally that didn't work too well because the benchmark suite could run Python 2 and Python 3 benchmarks simultaneously, but I don't know if that still works after the 2/3 merge.

As for having interpreter under test make the choice of what version to use is fine by me, it's just the current solution was the easiest to implement. If you want to change than go for it!

New changeset 88b6ef9aa9e9 by Antoine Pitrou in branch 'default':
Issue #19064: let perf.py decide which library path is required for which interpreter
http://hg.python.org/benchmarks/rev/88b6ef9aa9e9

Now done.