msg184421 - (view) |
Author: Dave Malcolm (dmalcolm) |
Date: 2013-03-18 04:08 |
Does the devguide document the benchmarking suite anywhere? I can't see it anywhere in the index on http://docs.python.org/devguide/ and google doesn't seem to show anything.
suggested content:
* how to run the benchmarks for a Python 2 implementation
* how to run the benchmarks for a Python 3 implementation
Sorry if it exists and I missed it
|
msg184670 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2013-03-19 20:28 |
No it does not, but it should.
|
msg185083 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-03-23 20:20 |
Mmmh... I'm not very warm on this one. If we start documenting the benchmark suite in the devguide, then people will start to be all gung ho about it, be obsessed with benchmark numbers, etc.
I guess what I'm saying is that *if* it's covered, then the very beginning of the section should explain that benchmarking is not important in most cases, and time spent on Python is generally better spent fixing bugs and adding features.
|
msg185103 - (view) |
Author: Ezio Melotti (ezio.melotti) * |
Date: 2013-03-23 22:19 |
Benchmarking is also useful to test that existing patches (that fix bug or add features) don't have a negative effect on the performance. Some kind of warning that advises against using benchmarks just to try and optimize everything sounds like a good idea though.
|
msg188251 - (view) |
Author: Meador Inge (meador.inge) * |
Date: 2013-05-02 00:49 |
I think this will be useful information. I tend to doing benchmarks
rather infrequently. Whenever I do them I forgot how I setup the suite
the previous time around.
In fact, I found this issue just now while googling for how to setup and
run the official benchmark suite :-)
|
msg193863 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2013-07-29 13:27 |
Documenting this would be useful because it appears to be non-trivial to actually run the benchmark suite with Python 3.
* Using "python perf.py -b default" fails due to some code in the
benchmarks that is not python3 compatible
* Using "make_perf3.sh" and then running "... -b default" still fails,
because not all libraries are translated (the test run stopped because
splitfire couldn't be found)
* Using "-b 2n3" instead instead of "-b default" or "-b all" works, but
runs only a subset of the benchmark.
|
msg193864 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-07-29 13:34 |
> * Using "-b 2n3" instead instead of "-b default" or "-b all" works,
> but
> runs only a subset of the benchmark.
I think we should include a copy of the 2to3'ed third-party libs in
the benchmarks repo, this would running avoid the length and tedious
conversion on every user's computer.
|
msg193865 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2013-07-29 13:40 |
The overhead of 2to3 isn't too bad, especially because only a subset of the benchmarks is translated in the first place.
It is more important that there is better documentation than there is now, it seems that you're supposed to use either '-b 2n3' or make_perf3.sh with '-b py3k'. Neither of which are mentioned in README.txt.
|
msg193866 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-07-29 13:45 |
> The overhead of 2to3 isn't too bad, especially because only a subset
> of the benchmarks is translated in the first place.
Well, the biggest time is spent translating the third-party libraries,
not benchmark files.
Also, it's simply tedious manual work (you have to create a separate
directory etc.).
> It is more important that there is better documentation than there is
> now, it seems that you're supposed to use either '-b 2n3' or
> make_perf3.sh with '-b py3k'. Neither of which are mentioned in
> README.txt.
That's because the benchmark suite was initially 2.x-only, and the
"make_perf3.sh" was a quick hack to make some of the benchmarks work
on 3.x.
Nowadays the goal is for benchmark files to be 2.x and 3.x compatible
in the same codebase, so that only third-party libs remain to
translate. Adding 3.x-compatible versions of the third-party libs
would eliminate that last step.
|
msg193868 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2013-07-29 14:01 |
The attached patch for the benchmark README.txt should document the status quo.
|
msg193871 - (view) |
Author: Dave Malcolm (dmalcolm) |
Date: 2013-07-29 15:04 |
On Mon, 2013-07-29 at 14:01 +0000, Ronald Oussoren wrote:
> Ronald Oussoren added the comment:
>
> The attached patch for the benchmark README.txt should document the status quo.
>
> ----------
> Added file: http://bugs.python.org/file31073/issue-18181-full-v3.txt
It looks like you attached the wrong patch: the patch you attached seems
to be unrelated to benchmarking (and presumably relates to issue 18181)
|
msg193872 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2013-07-29 15:10 |
Sorry about that. I stored the patch file in the wrong directory and attached the most recent file in the directory where I had expected the patch to be.
|
msg193876 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2013-07-29 15:38 |
I agree with Antoine that the time has come to drop make_perf3.sh and make it so that Python 3 doesn't require a separate build of the benchmark directory. Created http://bugs.python.org/issue18586 to track this idea.
|
msg216406 - (view) |
Author: Jeff Ramnani (jramnani) * |
Date: 2014-04-15 22:23 |
Now that bug #18586 is closed, could the Dev Guide point benchmarkers to the benchmarks repo and its README?
http://hg.python.org/benchmarks/file/9a1136898539/README.txt
|
msg216516 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2014-04-16 17:48 |
Jeff: yes it could :) Do you want to provide a patch?
|
msg219964 - (view) |
Author: Kavya Joshi (kavya) |
Date: 2014-06-07 19:35 |
I added a `Benchmarking` section in the dev guide with the relevant links and the intended use (testing rather than optimizing).
I also added an entry for the section in the Index.
Testing/verification:
I built the docs and visually inspected them.
|
msg221326 - (view) |
Author: Roundup Robot (python-dev) |
Date: 2014-06-22 23:49 |
New changeset 428b6350307e by Benjamin Peterson in branch 'default':
add a pointer to the benchmarks repo (closes #17449)
http://hg.python.org/devguide/rev/428b6350307e
|
|
Date |
User |
Action |
Args |
2022-04-11 14:57:42 | admin | set | github: 61651 |
2014-06-22 23:49:18 | python-dev | set | status: open -> closed
nosy:
+ python-dev messages:
+ msg221326
resolution: fixed stage: needs patch -> resolved |
2014-06-07 19:35:31 | kavya | set | files:
+ issue17449.patch
nosy:
+ kavya messages:
+ msg219964
keywords:
+ patch |
2014-04-16 17:48:13 | pitrou | set | messages:
+ msg216516 |
2014-04-15 22:23:22 | jramnani | set | nosy:
+ jramnani messages:
+ msg216406
|
2013-07-29 15:38:50 | brett.cannon | set | messages:
+ msg193876 |
2013-07-29 15:10:46 | ronaldoussoren | set | files:
+ issue-17449-readme.txt
messages:
+ msg193872 |
2013-07-29 15:09:13 | ronaldoussoren | set | files:
- issue-18181-full-v3.txt |
2013-07-29 15:04:11 | dmalcolm | set | messages:
+ msg193871 |
2013-07-29 14:01:31 | ronaldoussoren | set | files:
+ issue-18181-full-v3.txt
messages:
+ msg193868 |
2013-07-29 13:45:43 | pitrou | set | messages:
+ msg193866 |
2013-07-29 13:40:19 | ronaldoussoren | set | messages:
+ msg193865 |
2013-07-29 13:34:05 | pitrou | set | messages:
+ msg193864 |
2013-07-29 13:27:19 | ronaldoussoren | set | nosy:
+ ronaldoussoren messages:
+ msg193863
|
2013-05-29 19:34:08 | eric.snow | set | nosy:
+ eric.snow
|
2013-05-02 00:49:34 | meador.inge | set | nosy:
+ meador.inge messages:
+ msg188251
|
2013-04-06 02:16:48 | edmond.burnett | set | nosy:
+ edmond.burnett
|
2013-03-23 22:19:44 | ezio.melotti | set | messages:
+ msg185103 |
2013-03-23 20:20:17 | pitrou | set | nosy:
+ pitrou messages:
+ msg185083
|
2013-03-23 14:14:43 | ezio.melotti | set | type: enhancement stage: needs patch |
2013-03-19 20:28:03 | brett.cannon | set | keywords:
+ easy assignee: brett.cannon -> messages:
+ msg184670
|
2013-03-18 04:08:17 | dmalcolm | create | |