The commit dcfcd14 of bpo-35766 added a new cf_feature_version field to PyCompilerFlags. Each PyCompilerFlags variable must properly initialize this new field to PY_MINOR_VERSION. I propose to add a new _PyCompilerFlags_INIT macro to statically initialize such variable. I'm not sure if it should be public or not.

The PyCompilerFlags structure is excluded from the stable ABI (PEP-384), but it's documented in the "The Very High Level Layer" C API documentation:
https://docs.python.org/dev/c-api/veryhigh.html#c.PyCompilerFlags

Structure fields are documented there:

struct PyCompilerFlags {
    int cf_flags;
}

The doc is outdated. I'm not sure if it's on purpose or not.

Moreover, the new PyCompilerFlags.cf_feature_version field is not documented in https://docs.python.org/dev/whatsnew/3.8.html#changes-in-the-c-api whereas C extensions using PyCompilerFlags should initialize cf_feature_version to PY_MINOR_VERSION?

I'm not sure if C extensions really *must* initialize cf_feature_version, since the field is only used if PyCF_ONLY_AST flag is set in the cf_flags field.

Something else, ast.parse() has been modified to use a (major, minor) version tuple rather an integer to specify the Python version in feature_version, but PyCompilerFlags still only uses the minor major. This API will be broken once the Python major version will be increased to 4, no? Would it make sense to use PY_VERSION_HEX format which includes the major version instead?

Or cf_feature_version could be a structure with major and minor fields, but that might be overkill?

Something else, ast.parse() has been modified to use a (major, minor) version tuple rather an integer to specify the Python version in feature_version, but PyCompilerFlags still only uses the minor major. This API will be broken once the Python major version will be increased to 4, no? Would it make sense to use PY_VERSION_HEX format which includes the major version instead?

I can work on a PR to change cf_feature_version format, but I would prefer to agree here on what is the best format ;-)

Note: compile() has a private keyword-only _feature_version which is also the Python minor version (int): don't include the major version. If we change cf_feature_version, we may also change compile(_feature_version=N) format.

It's fine to document the current state. I don't think you should spend any time *changing* the API to "future-proof" it.

I am hoping that larger changes to the compiler implementation will happen before Python 4, which will make the whole API moot (including the "parser" module, which should be deprecated ASAP). The compiler is excluded from the ABI for a reason.

New changeset 2c9b498 by Victor Stinner in branch 'master':
bpo-37253: Document PyCompilerFlags.cf_feature_version (GH-14019)
2c9b498

New changeset 37d66d7 by Victor Stinner in branch 'master':
bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)
37d66d7

New changeset a04ea4f by Victor Stinner in branch 'master':
bpo-37253: Fix typo in PyCompilerFlags doc (GH-14036)
a04ea4f

Many PyRun_xxx() functions of the public C API accept a "PyCompilerFlags* flags" parameter. So yeah, I documented the new cf_feature_version. PyCompilerFlags structure is kind of public ;-)

New changeset 92e836c by Miss Islington (bot) in branch '3.8':
bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)
92e836c

New changeset f9445a3 by Victor Stinner in branch '3.8':
[3.8] bpo-37253: Document PyCompilerFlags.cf_feature_version (GH-14019) (GH-14038)
f9445a3

New changeset 022ac0a by Victor Stinner in branch 'master':
bpo-37253: Remove PyAST_obj2mod_ex() function (GH-14020)
022ac0a

New changeset 032bf30 by Miss Islington (bot) in branch '3.8':
bpo-37253: Remove PyAST_obj2mod_ex() function (GH-14020)
032bf30

It's fine to document the current state. I don't think you should spend any time *changing* the API to "future-proof" it.

Ok.

I am hoping that larger changes to the compiler implementation will happen before Python 4, which will make the whole API moot (including the "parser" module, which should be deprecated ASAP). The compiler is excluded from the ABI for a reason.

Aha, that sounds exciting :-)

I created bpo-37268 to propose to deprecate the parser module in Python 3.8.

--

PyCompilerFlags changes are now documented. I made the small code changes that I wanted to do in 3.8 and master. I close the issue.