Currently, the compiler starts with a list of arrays of instructions, packs them to 1/3/6-bytes-apiece bytecodes, fixes up all the jumps, and then calls PyCode_Optimize on the result. This makes the peephole optimizer much more complicated. Assuming PEP 511 is accepted, it will also make plug-in bytecode optimizers much more complicated (and probably wasteful--they'll each be repeating the same work to re-do the fixups).

The simplest alternative (as suggested by Serhiy on -ideas) is to expose an "unpacked" bytecode to the optimizer (in the code parameter and return value and lnotab_obj in-out parameter for PyCode_Optimize, and similarly for PEP 511) where each instruction takes a fixed 4 bytes. This is much easier to process. After the optimizer returns, the compiler packs opcodes into the usual 1/3/6-byte format, removing NOPs, retargeting jumps, and adjusting the lnotab as it goes. (Note that it already pretty much has code to do all of this except the NOP removal; it's just doing it before the optimizer instead of after.)

Negatives:

 * Arguments can now only go up to 2**23 instead of 2**31. I don't think that's a problem (has anyone ever created a code object with 4 million instructions?).

 * A bit more work for the compiler; we'd need to test to make sure there's no measurable performance impact.

We could also expose this functionality through C API PyCode_Pack/Unpack and Python dis.pack_code/unpack_code functions (and also make the dis module know how to parse unpacked code), which would allow import hooks, post-processing decorators, etc. to be simplified as well. This would remove some, but not all, of the need for things like byteplay. I think this may be worth doing, but I'm not sure until I see how complicated it is.

We could even allow code objects with unpacked bytecode to be executed, but I think that's unnecessary complexity. Nobody should want to do that intentionally, and if an optimizer lets such code escape by accident, a SystemError is fine.

MRAB implied an alternative: exposing some slightly-higher-level label-based format. That would be even nicer to work with. But it's also more complicated for the compiler and for the API, and I think it's already easy enough to handle jumps with fixed-width instructions.

Reading more about wpython (slide 23 of https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/wpython2/Cleanup%20and%20new%20optimizations%20in%20WPython%201.1.pdf), one of his optimizations was moving the peephole optimizer into the compiler, so it could just use the linked list of block objects of arrays of instruction objects, instead of raw bytecode.

Obviously that idea isn't compatible with PEP 511. But on the off chance that PEP 511 founders, that might be the simplest answer to this problem.

PEP 511 was rejected.
The "peephole" optimizer now operates on the internal IR, not the bytecode.

> The "peephole" optimizer now operates on the internal IR, not the bytecode.

Python/ast_opt.c is cool ;-) Thanks INADA-san for creating it!