Right now, opcode information is duplicated in Include/opcode.h and Lib/opcode.py. I should only have to update one manually and have the other one generated automatically. opcode.h should probably be generated by a script from opcode.py.

Here is a simple script which prints the opcode.h in console. We can redirect it as required.

Second version of the script with the fix for HAVE_ARGUMENT

Third revision with fixed empty spaces at the end of the lines.

As we discussed on #python-dev channel this new patch includes the script in Tools/scripts/generate_opcode_h.py and it also contains the required Makefile.pre.in change so that it gets auto(re)generated at compile time if required.

Version 2 of the patchset edited as per review.

Patch looks good to me.

Please add the file to .hgtouch as well, and verify that "make touch" works correctly.

New patch with .hgtouch file details.

New patch with proper changesets.

New patch from a clean repo, with changed opcode.h

New patch with changes as suggested by Brett.

New changeset 56e7fa27f979 by Kushal Das in branch 'default':
Closes Issue 17861: Autogenerate Include/opcode.h from opcode.py.
http://hg.python.org/cpython/rev/56e7fa27f979

FYI, this broke building in a separate object directory (again!) for multiple reasons: it's running the script without specifying $(srcdir), and it's writing to $(srcdir)/Include/opcode.h (where $(srcdir) may be unwritable), and it's picking up the wrong opcode module: the sys.path mucking in Tools/generate_opcode_h.py isn't inserting the right directory for 'import opcode' to pick up the target-Python's opcode.py. It should probably be using execfile() and have the name of the file passed in, instead.

Even if it were importing the right opcode.py, relying on 'import' or 'execfile' here sounds like a bad idea: it means the opcode module of the *target* Python needs to be compatible with the Python that the generate_opcode_h.py script runs with. Since the syntax of opcode.h is much more constrained, a more flexible approach, in my opinion, would be to parse opcode.h to produce opcode.py (or an _opcode.py with just the definitions, that opcode.py would then import.)

Here's a minimal patch to at least make the current mechanism work when using a separate build directory.

I still don't like the fact that this is importing opcode.py in a different Python than the target Python. Nor do I like that the script hardcodes information about the opcodes (specifically, the first opcode to HAVE_ARGUMENT.) This feels like it's not actually better for maintenance than the previous solution.

New changeset 2b187c9e3e92 by Thomas Wouters in branch 'default':
Fix Tools/scripts/generate_opcode_h.py from issue #17861 to work correctly
http://hg.python.org/cpython/rev/2b187c9e3e92

This generator script breaks the daily OSX DMG builds on my bolen-dmg buildslave for the 3.x branch.

The issue is with the use of "with" as the slave uses Python 2.5 to build the installer.  Now, that's old, and I'm not even sure how necessary the daily builds are to keep running, but I believe the Mac build-installer script still tries to target 2.5 as a minimum - though I'm not sure if that's true for all build tools.  But the fix is fairly simple (a __future__ import).

See attached patch.

Alternatively, if the minimum Python to build the installer should be bumped, I can work on that for the slave.

New changeset 1fd9c3f6cf68 by Ned Deily in branch 'default':
Issue #17861: Allow generate_opcode_h to run with a system Python 2.5.
http://hg.python.org/cpython/rev/1fd9c3f6cf68

Thanks, David. Ideally, the generator script shouldn't run during an installer build since presumably the generated file should be up-to-date in the repo.  "make touch" could handle that but the installer build does use a separate build/object directory and doesn't currently use "make touch".  For Python 3.5, the installer build now does need at least a Python 2.6 to build the documentation but there's no reason to unnecessarily break the main build with such a simple fix available.

It would be possible to have the daily DMG builder run "make touch".

Martin, it could if "make touch" worked when building outside of the source directory (Issue21383).

Currently an explicit `make regen-opcode` and `make regen-opcode-targets` (or just `make regen-all`) are needed for regenerating C files from opcode.py.

Is anything left to do in this issue?