This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author abarnert
Recipients abarnert, benjamin.peterson, georg.brandl, pitrou, serhiy.storchaka, vstinner, yselivanov
Date 2016-02-05.22:17:43
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1454710664.23.0.316699327961.issue26300@psf.upfronthosting.co.za>
In-reply-to
Content
Currently, the compiler starts with a list of arrays of instructions, packs them to 1/3/6-bytes-apiece bytecodes, fixes up all the jumps, and then calls PyCode_Optimize on the result. This makes the peephole optimizer much more complicated. Assuming PEP 511 is accepted, it will also make plug-in bytecode optimizers much more complicated (and probably wasteful--they'll each be repeating the same work to re-do the fixups).

The simplest alternative (as suggested by Serhiy on -ideas) is to expose an "unpacked" bytecode to the optimizer (in the code parameter and return value and lnotab_obj in-out parameter for PyCode_Optimize, and similarly for PEP 511) where each instruction takes a fixed 4 bytes. This is much easier to process. After the optimizer returns, the compiler packs opcodes into the usual 1/3/6-byte format, removing NOPs, retargeting jumps, and adjusting the lnotab as it goes. (Note that it already pretty much has code to do all of this except the NOP removal; it's just doing it before the optimizer instead of after.)

Negatives:

 * Arguments can now only go up to 2**23 instead of 2**31. I don't think that's a problem (has anyone ever created a code object with 4 million instructions?).

 * A bit more work for the compiler; we'd need to test to make sure there's no measurable performance impact.

We could also expose this functionality through C API PyCode_Pack/Unpack and Python dis.pack_code/unpack_code functions (and also make the dis module know how to parse unpacked code), which would allow import hooks, post-processing decorators, etc. to be simplified as well. This would remove some, but not all, of the need for things like byteplay. I think this may be worth doing, but I'm not sure until I see how complicated it is.

We could even allow code objects with unpacked bytecode to be executed, but I think that's unnecessary complexity. Nobody should want to do that intentionally, and if an optimizer lets such code escape by accident, a SystemError is fine.

MRAB implied an alternative: exposing some slightly-higher-level label-based format. That would be even nicer to work with. But it's also more complicated for the compiler and for the API, and I think it's already easy enough to handle jumps with fixed-width instructions.
History
Date User Action Args
2016-02-05 22:17:44abarnertsetrecipients: + abarnert, georg.brandl, pitrou, vstinner, benjamin.peterson, serhiy.storchaka, yselivanov
2016-02-05 22:17:44abarnertsetmessageid: <1454710664.23.0.316699327961.issue26300@psf.upfronthosting.co.za>
2016-02-05 22:17:44abarnertlinkissue26300 messages
2016-02-05 22:17:43abarnertcreate