The commit dcfcd146f8e6fc5c2fc16a4c192a0c5f5ca8c53c of bpo-35766 added a new cf_feature_version field to PyCompilerFlags. Each PyCompilerFlags variable must properly initialize this new field to PY_MINOR_VERSION. I propose to add a new _PyCompilerFlags_INIT macro to statically initialize such variable. I'm not sure if it should be public or not.

The PyCompilerFlags structure is excluded from the stable ABI (PEP 384), but it's documented in the "The Very High Level Layer" C API documentation:
https://docs.python.org/dev/c-api/veryhigh.html#c.PyCompilerFlags

Structure fields are documented there:

struct PyCompilerFlags {
    int cf_flags;
}

The doc is outdated. I'm not sure if it's on purpose or not.

Moreover, the new PyCompilerFlags.cf_feature_version field is not documented in https://docs.python.org/dev/whatsnew/3.8.html#changes-in-the-c-api whereas C extensions using PyCompilerFlags should initialize cf_feature_version to PY_MINOR_VERSION?

I'm not sure if C extensions really *must* initialize cf_feature_version, since the field is only used if PyCF_ONLY_AST flag is set in the cf_flags field.

Something else, ast.parse() has been modified to use a (major, minor) version tuple rather an integer to specify the Python version in feature_version, but PyCompilerFlags still only uses the minor major. This API will be broken once the Python major version will be increased to 4, no? Would it make sense to use PY_VERSION_HEX format which includes the major version instead?

Or cf_feature_version could be a structure with major and minor fields, but that might be overkill?

> Something else, ast.parse() has been modified to use a (major, minor) version tuple rather an integer to specify the Python version in feature_version, but PyCompilerFlags still only uses the minor major. This API will be broken once the Python major version will be increased to 4, no? Would it make sense to use PY_VERSION_HEX format which includes the major version instead?

I can work on a PR to change cf_feature_version format, but I would prefer to agree here on what is the best format ;-)

Note: compile() has a private keyword-only _feature_version which is also the Python minor version (int): don't include the major version. If we change cf_feature_version, we may also change compile(_feature_version=N) format.

It's fine to document the current state. I don't think you should spend any time *changing* the API to "future-proof" it.

I am hoping that larger changes to the compiler implementation will happen before Python 4, which will make the whole API moot (including the "parser" module, which should be deprecated ASAP). The compiler is excluded from the ABI for a reason.

New changeset 2c9b498759f4fc74da82a0a96d059d666fa73f16 by Victor Stinner in branch 'master':
bpo-37253: Document PyCompilerFlags.cf_feature_version (GH-14019)
https://github.com/python/cpython/commit/2c9b498759f4fc74da82a0a96d059d666fa73f16

New changeset 37d66d7d4bc7dbac9809d69966a774ebb32563be by Victor Stinner in branch 'master':
bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)
https://github.com/python/cpython/commit/37d66d7d4bc7dbac9809d69966a774ebb32563be

New changeset a04ea4f92ccbe20ffdbb5fa9b6bb93ee8da50f5d by Victor Stinner in branch 'master':
bpo-37253: Fix typo in PyCompilerFlags doc (GH-14036)
https://github.com/python/cpython/commit/a04ea4f92ccbe20ffdbb5fa9b6bb93ee8da50f5d

Many PyRun_xxx() functions of the public C API accept a "PyCompilerFlags* flags" parameter. So yeah, I documented the new cf_feature_version. PyCompilerFlags structure is kind of public ;-)

New changeset 92e836c7dcaf74f7b8617250414224d24d1eb1f2 by Miss Islington (bot) in branch '3.8':
bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)
https://github.com/python/cpython/commit/92e836c7dcaf74f7b8617250414224d24d1eb1f2

New changeset f9445a391e6a91103fbe88bff4d3adc07f526c73 by Victor Stinner in branch '3.8':
[3.8] bpo-37253: Document PyCompilerFlags.cf_feature_version (GH-14019) (GH-14038)
https://github.com/python/cpython/commit/f9445a391e6a91103fbe88bff4d3adc07f526c73

New changeset 022ac0a497b668d8b15e34e582a6396ead1a35e1 by Victor Stinner in branch 'master':
bpo-37253: Remove PyAST_obj2mod_ex() function (GH-14020)
https://github.com/python/cpython/commit/022ac0a497b668d8b15e34e582a6396ead1a35e1

New changeset 032bf30643fff49b5595b53f9c1ce5b2cd2df504 by Miss Islington (bot) in branch '3.8':
bpo-37253: Remove PyAST_obj2mod_ex() function (GH-14020)
https://github.com/python/cpython/commit/032bf30643fff49b5595b53f9c1ce5b2cd2df504

> It's fine to document the current state. I don't think you should spend any time *changing* the API to "future-proof" it.

Ok.

> I am hoping that larger changes to the compiler implementation will happen before Python 4, which will make the whole API moot (including the "parser" module, which should be deprecated ASAP). The compiler is excluded from the ABI for a reason.

Aha, that sounds exciting :-)

I created bpo-37268 to propose to deprecate the parser module in Python 3.8.

--

PyCompilerFlags changes are now documented. I made the small code changes that I wanted to do in 3.8 and master. I close the issue.