Author rbcollins
Recipients alex, arigo, fijall, rbcollins, rhettinger, serhiy.storchaka
Date 2015-03-17.21:32:11
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1426627931.33.0.39926122895.issue23552@psf.upfronthosting.co.za>
In-reply-to
Content
I think for PyPI its actually important here - the JIT'd state of the code is essentially global state being mutated - you can't assess how fast the code is without first warming up the JIT, and if it warms up half way through your fastest run, you're still not actually finding out what you might want to find out.

E.g. do you want to know:
 - how fast is this unjitted [e.g. CLI's]
 - how fast will this be once its hot [e.g. services]

Personally, I think as a first approximation, warning about massive variance is a good thing. We could add an option to turn it off, and we could also look at hooking properly into the jit to allow detection of stable state and benchmark only from there on in. But those extra things don't detract from the utility of warning about suspect runs.
History
Date User Action Args
2015-03-17 21:32:11rbcollinssetrecipients: + rbcollins, arigo, rhettinger, alex, fijall, serhiy.storchaka
2015-03-17 21:32:11rbcollinssetmessageid: <1426627931.33.0.39926122895.issue23552@psf.upfronthosting.co.za>
2015-03-17 21:32:11rbcollinslinkissue23552 messages
2015-03-17 21:32:11rbcollinscreate