This command will contain all options that are used to build extensions (out of the build* and install* commands) and will create a confifuration file.

build* and install* commands will be able to read back the file if present, and use its values, so we don't have to pass the options again.

See 4Suite configure command for an example

My review of 4Suite’s config command is on <http://bitbucket.org/Merwok/distutils2/wiki/4suite/config>. Comments welcome.

(Adding my mentor to the nosy list.)

Distutils2 already has a config command used to configure some aspects of C compilation. Not sure if I should expand that or write an unrelated configure command.

start a new command from scratch. The configure command is not dealing but with generating a configuration file.

config is implementing a whole compilation chain.

I have a working configure command at http://bitbucket.org/Merwok/distutils2-configure which works. (Read my short reports on https://wokslog.wordpress.com/ for more details.) It’s not completely finished, though:

1) I import build and install to get their user_options, to avoid duplicating these lines. Is that bad?

2) Test coverage is not exhaustive. This will be done incrementally.

3) I tried directly using Distribution and configure objects in the tests, but couldn’t get them to work; I temporarily settled for running Python (sys.executable) in a subprocess. I’m looking for help to change that.

4) As it is now, the configure command just records options without checking for conflicts (e.g. using both –prefix and –user for install). While not in the original description of the task, I think that it should be added. Prior art like 4Suite’s config command or a prototype for configure done by Tarek that I discovered only today support this idea. Is that okay? I’ll have to avoid code duplication between configure, build and install.

5) Should the command write different files per Python version and platform? 4Suite’s config command does that.

Shouldn't "config" and "configure" have more distinguishable names?
(if I understand correctly, they both are distutils2 commands)

4Suite’s DistExt has a config command similar to the new Distutils2 configure command described in this bug report. The Distutils config command is an unfinished command with use similar to autotools configure scripts. From the module docstring:

“a (mostly) empty command class that exists mainly to be sub-classed by specific module distributions and applications.  The idea is that while every "config" command is different, at least they're all named the same, and users always see "config" in the list of standard commands.  Also, this is a good place to put common configure-like tasks: "try to compile this C code", or "figure out where this header file lives"”.

The module has seen 26 commits in 10 years. I find the name clash confusing too for end users and wanted to bring the issue later. If I understand autotools correctly, it encompasses both config and configure commands. Since there are no conflicts in options or functionality (config.run is a no-op, configure.run writes a file), I think dropping config or merging it with configure would make a lot of sense. But first things first, configure needs to be reviewed and accepted (5 open questions :) and then I’ll propose a merge into a unified config command (short names win).

Quick update: I have done a basic part of this, but Tarek explained to me the real usefulness of this command, i.e. provide build and installation options for third-party code (for the same purposes that get_config_var in Python’s sysconfig). I am currently working on #8252 but do not forget this one.

Tarek also answered my questions.

1) 4) build and install options should be defined and checked only once, in configure. build and install would inherit from this class and get their option from it.

3) I switched to using distutils2.core.run_setup, which Tarek said is okayish. Understanding why the usual idiom of using the classes directly doesn’t work will be better.

5) Since the cache file (the file that contains options, written by configure) is tied to a precise build, it should be specific. I can put the platform and Python version in the filename, or maybe put the file in the platform-specific build directory. Not sure about the naming in dists. I still have to think about the overlap between setup.cfg and configure.cache.

WRT #5, it would be possible to use section headers within the configure.cache file to indicate a particular "platform build".  Very much like how 4Suite's `config` command stored the cache file.

Adding message from jkloth removed by error (remove/reply confusion):

for #1 instead of importing `build`/`install` commands for their options, why not define the options only on the `configure` command?  The B/I commands could just "lookup" their needed options from the configure command.

On that note, how does the B/I commands get the options defined by `configure`? A quick look at the code doesn't show them using the "configured" options.

> why not define the options only on the `configure` command?

This is what Tarek told me and what I thought I said two messages earlier.

> On that note, how does the B/I commands get the options defined by `configure`?

See http://bitbucket.org/Merwok/distutils2-configure/src/tip/src/distutils2/dist.py#cl-363 and line 384. Nothing to change in the commands, just make Distribution pick values from the cache file at the right time (after regular config files and before command-line options).

How would the build or install command get the options in this case:

python setup.py configure --prefix=$HOME build install

Remember that commands can be chained (and that commands can run other commands).  I would like to be able to do (and can do right now):

python setup.py install

and have both `configure` and `build` be run with their defaults.

configure is not config. As I understand it, the config command aimed to be something like ./configure in autotools, thus usable in combination with build and install. configure is different: it is meant to be run once to create a configure.cache file which saves possibly long and complicated options. IOW, its goal was to be typed only once, not in combination with build or install. This is why build does not call configure. I have written a short doc to explain that, see configure.rst in my repo or the built version at http://distutils2.notmyidea.org/configure.html

This message and the linked doc explain the current behavior of configure as I understood its purpose a month ago, IMO respecting what Tarek wrote in the bug report. However, Tarek he did not explain all that he had in mind here, like I wrote here on 2010-07-17 (I don’t have notes on that :/ so I’ll have to ask him again and sort out the setup.cfg/configure.cache overlap). When I get back to configure, the command line “configure --prefix ~ install” will work according to your expectations.

Status update.

* configure works: it checks options (like install does) and writes to a file.

* New design for build: It uses set_undefined_options to get options from configure (previously it read the cache file, but this caused nasty issues; I’ll document those designs some day, it was instructional for me).  All tests pass.

* Same thing for install, but it’s harder to make it work.  install.finalize_options checks for conflicts (for example you can’t set --user and --prefix=x) and then fills all options (so prefix gets a value).  Now configure has the same checks, so that it does not write non-working options to the cache file.  The problem is that all options are set in configure.finalize_install_options and then passed to install with set_undefined_options, and then install.finalize_options shouts at you because you can’t set all options.  I tried a simple and stupid fix for that (disable option conflict check when options come from configure), which fixed all configure tests but caused test_core to fail.  This means that while the configure tests passed, they were only superficially working; I have to be suspicious to my own tests and out general test coverage.

(BTW I haven’t pushed the work in progress yet but I will shortly.)

I have turned my branch into a clean patch for default: https://bitbucket.org/Merwok/distutils2/changeset/076b15559290

Notes:

- You can diff configure.py against build.py and install_dist.py from an earlier revision to see what code has been moved where.

- Two tests for install_dist still fail, but this could be only the tip of the iceberg.  We need more tests there.  Installation paths is a tricky business, and we have to make sure configure does not break anything.

- I have changed install to install_dist in the code, tests and docs, but only in the part of the docs I added for configure.  The rest of the file is inaccurate, but that’s outside the scope of this patch.

- Some to-do features: distribute the configure.cache file in the dist-info directory (requires adding a hook system to install_distinfo); add info about the compiler type; probably more.