build command of packaging should accept --compile, --no-compile and --optimize options and pass them to build_py command.

Thinking about this, build_py --compile clearly refers to byte-compilation of Python modules, but the same option on the build command is more ambiguous: It could be interpreted to mean “skip C compilation”.

Maybe --byte-compile and --no-byte-compile could be used.

Juste a note: currently, when a command sets one option from the value given to another command, they have the same name, with two kinds of exceptions:
- build --build-lib becomes build_lib --build-dir (etc.)
- install --install-lib becomes install_lib --install-dir (etc.)

So, introducing --(no-)byte-compile and --optimize-bytecode= would change that scheme, but it is not a big deal.

On the third hand, maybe this is an opportunity to change the option names on the build_py and install* commands too.  For example, for a long time I thought that --optimize implied --compile, but actually they’re distinct options.

BTW, would these new options simplify some scripts you have, or do you just request them for consistency?

Byte-compilation should be disabled during building of packages in Gentoo. PYTHONDONTWRITEBYTECODE="1" is set by default in environment. This variable affects distutils and until recently it affected packaging. --no-byte-compile will have to be explicitly passed to `pysetup${version} run build` and `pysetup${version} run install_dist` to disable byte-compilation.

Okay, I think it’s a valuable use case.

(Out of curiosity, why don’t you want byte-compiled files on your system?  It speeds up imports, and problems due to the presence of stray pyc files when the py is deleted are gone in 3.2+.  Maybe you have custom tools to byte-compile, like Debian?)

Do you have any opinion about my renaming suggestion?
Without renaming, we’d have that:

  build --byte-compile --no-byte-compile --optimize-bytecode=[012]
  build_py --compile --no-compile --optimize=[012]

If we want to use the same name and make the names clearer, we could have:

  build(_py) --compile-pyc --no-compile-pyc --compile-pyo=[012]

I suggest to not use "pyc" and "pyo" in options, because ".pyc" and ".pyo" filename extensions are specific to a subset of Python implementations. Jython uses "$py.class" filename extension (module$py.class for module.py).

You could use --byte-compile, --no-byte-compile and --optimize-bytecode (or --optimize-byte-code) for both build and build_py commands.

Do settings from setup.cfg affect byte-compilation?

> I suggest to not use "pyc" and "pyo" in options, because ".pyc" and ".pyo" filename extensions
> are specific to a subset of Python implementations. Jython uses "$py.class" filename extension
> (module$py.class for module.py).
But these are extension modules, not Python modules byte-compiled for caching, are they?

> You could use --byte-compile, --no-byte-compile and --optimize-bytecode (or --optimize-byte-code)
> for both build and build_py commands.
I like the first two names, but still think that --optimize-bytecode might make people imply that the option is dependent on (or implying) --byte-compile.  Maybe it’s just me; I did not read the source for these options closely and was under this misconception for months.  OTOH, even with one maybe possibly ambiguous option name, I can still make the help text clearer (i.e. “byte-compile Python modules” and “byte-compile Python modules with optimizations”).

(We’re spending a lot of thoughts for a very marginally useful feature (.pyo files).)

> Do settings from setup.cfg affect byte-compilation?
Yes.  Like any command options, --(no-)compile and --optimize can be given in config files or on the command-line.

BTW, why don’t you want byte-compiled files on your system?

Jython's *$py.class files are byte-compiled modules, not extension modules.

There should be a way to disable generation of *.pyo files on command line even if setup.cfg enables it.

IMHO it would make more sense if --optimize-bytecode was dependant on --byte-compile option:
--no-byte-compile                      -> No *.pyc and *.pyo
--byte-compile --optimize-bytecode=0   -> Only *.pyc
--byte-compile --optimize-bytecode=0,1 -> *.pyc and *.pyo (with docstrings)
--byte-compile --optimize-bytecode=0,2 -> *.pyc and *.pyo (without docstrings)
--byte-compile --optimize-bytecode=1   -> Only *.pyo (with docstrings)
--byte-compile --optimize-bytecode=2   -> Only *.pyo (without docstrings)
--byte-compile --optimize-bytecode=1,2 -> Error

Byte-compiled files in Gentoo are generated separately, after installation.

> Jython's *$py.class files are byte-compiled modules, not extension modules.
Thanks for the data point.  Agreed distutils[2] should not say “pyc” and “pyo” then.

> There should be a way to disable generation of *.pyo files on command line even if
> setup.cfg enables it.
There is.  The precedence of options is: stdlib distutils.cfg < (overriden by) user .pydistutils.cfg < local setup.cfg < options on the command line.  Pass --no-compile --optimize=0 to never ever byte-compile (or pass --no-user-cfg and rely on the defaults).

> IMHO it would make more sense if --optimize-bytecode was dependant on --byte-compile option:
It was also my expectation, as I told.  The scheme that you propose keeps all current possibilities, it’s nice!  What do you think about conflating two options into one?

> --no-byte-compile         -> No *.pyc or *.pyo
> --byte-compile            -> Only *.pyc
> --byte-compile=0          -> Only *.pyc
> --byte-compile=0,1        -> *.pyc and *.pyo (level 1)
(etc.)
There may be a technical hurdle to overcome (not sure the option parsing system will allow 0 or more arg), but I’m asking for human interface feedback first.  (I’m just trying to make the list of options a bit smaller to reduce the overload, but if it feels complicated I won’t do it.)

> Byte-compiled files in Gentoo are generated separately, after installation.
Are you using standard py_compile or compileall modules or your own scripts?  I’ve seen that Debian for example has its own scripts and I’m sad to see no feature requests upstreamed to us instead.

--byte-compile=arguments is a good idea.

(Gentoo uses py_compile and compileall modules.)

Thanks for all the feedback!  So, for anyone interested in contributing, two patches are needed:

One that changes the existing options to use the new names and parsing behavior (--no-compile, --compile[=0,1,2]) and adds tests for the erroneous --compile=1,2 (other combinations are already covered).

A second one to add --no-compile and --compile to build, make build_py take its default values from build, and add tests to see if this works.

As usual, I’m available for help here, on the core-mentorship list or in private email.  (I don’t plan to do this myself in the short term because it’s a minor feature, I have an alpha to release and I have to keep the easiest things for new contributors.)

Here is the last proposition of Eric done at the last Montreal Sprint (May 12):

--no-byte-compile    -> No *.pyc and *.pyo
--byte-compile=b     -> Only *.pyc
--byte-compile=b,o   -> *.pyc and *.pyo (with docstrings)
--byte-compile=b,oo  -> *.pyc and *.pyo (without docstrings)
--byte-compile=o     -> Only *.pyo (with docstrings)
--byte-compile=oo    -> Only *.pyo (without docstrings)
--byte-compile=o,oo  -> Error

The parameters are symetric with the -B, -o and -oo of the python command.

FTR I asked Julien to provide his opinion as a user and he agreed that having just one --byte-compile/--no-byte-compile option pair seemed better than the existing names (where --compile and --optimize are unrelated).

I wanted to avoid --byte-compile=0 to generate pyc, because I expected people to interpret it as “disabled” instead of “optimization level 0”.  Thus the idea of using letters (and avoiding to use c and o as not all VMs have pyc and pyo files).

Note that the new spelling for --compile is the seemingly redundant --byte-compile=b, but that does not matter as it will be the default value so people won’t type it.

One thing just came to my mind: if we change option names and people use the new names in setup.cfg or ~/.pydistutils.cfg, then the file will stop being compatible with distutils.

It is possible to only change the "frontend" (options: byte-compile, compile, no-byte-compile, optimize) without changing the "backend" (attributes: compile and optimize). This way it is then easy to handle both sets of options (no-byte-compile, byte-compile, compile and optimize) for as long as it is necessary. I wonder if that could cause any problem...