classification
Title: pybench: Cannot compare 2.x and 3.x benchmarks
Type: behavior Stage: resolved
Components: Demos and Tools Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: 5180 Superseder:
Assigned To: lemburg Nosy List: belopolsky, ezio.melotti, flox, iritkatriel, lemburg, pitrou
Priority: normal Keywords:

Created on 2010-06-28 15:42 by belopolsky, last changed 2021-06-17 10:50 by lemburg. This issue is now closed.

Messages (4)
msg108834 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-06-28 15:41
There are several issues that prevent

$ python3 Tools/pybench/pybench.py -s 27.bench -c 32.bench

from working.

1. There is a bug in _pickle that prevents 2.x pickles that contain classic class instances from loading in 3.x.  This is addressed in issue5180.

2. Once _pickle issue is addresses, 2.7 benchmark files still cannot be loaded because they contain references to the Unicode tests and Unicode.py is not loadable in python3.

3. Once Unicode.py gets 2to3 treatment, we are still left with 

* Benchmark versions differ: cannot compare this benchmark to "32.bench" !


The last issue is probably the simplest.  Version number was bumped up in r69411, and a log entry says that the intent was to keep pybench 2.1 compatible with 2.x.  Indeed, pybench 2.1 from py3k runs under trunk python and resulting benchmark file can be compared with 3.x generated benchmark once issues 1 and 2 above are resolved.  It may, however be worth consideration to synchronize pybench versions shipped with 27 and 32.  I understand that this is likely to be too late for 2.7, but I am checking 2.7 version in case this is considered appropriate for 2.7.1.

The Unicode.py issue was discussed under issue5180 in msg108819 and msg108821.  I like MAL's suggestion of Strings3 module, but I think it would be useful to be able to compare 2.x str performance to both 3.x str and 3.x bytes.  Similarly, I think String3 should have from __future__ import unicode_literals and be loadable in 2.7/2.6 and server as means to compare 2.x and 3.x unicode performance.

For the _pickle issue, I am about to attach a patch to issue5180.
msg222691 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-10 19:28
@Alexander do you want to pick this up again?
msg395991 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-17 10:45
Can we close this? I don't think we are still interested in performance comparisons with Python 2.
msg395992 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2021-06-17 10:50
Yes, closing.
History
Date User Action Args
2021-06-17 10:50:02lemburgsetstatus: pending -> closed

messages: + msg395992
stage: needs patch -> resolved
2021-06-17 10:45:46iritkatrielsetstatus: open -> pending

nosy: + iritkatriel
messages: + msg395991

resolution: out of date
2019-04-26 18:51:57BreamoreBoysetnosy: - BreamoreBoy
2014-07-10 19:28:01BreamoreBoysetnosy: + BreamoreBoy

messages: + msg222691
versions: + Python 3.5, - Python 3.3
2012-07-16 13:34:24floxsetversions: + Python 3.4, - Python 3.2
2011-11-16 15:03:34floxsetnosy: + flox
2011-11-16 11:29:49ezio.melottisetnosy: + ezio.melotti

versions: + Python 3.3
2010-10-21 13:30:37georg.brandlsetassignee: lemburg
2010-06-28 16:08:22belopolskysetdependencies: + 3.1 cannot unpickle 2.7-created pickle
2010-06-28 15:42:01belopolskycreate