I found one error in Doc/data/refcounts.dat (see issue9369, second argument of PyObject_CallMethodObjArgs is `PyObject*`, not `char*`). Perhaps there are other errors. How this file was generated first? Is it possible to write a tool which will automatically verify at least signatures of functions?

New changeset 0889ab0d0da1 by Serhiy Storchaka in branch '3.3':
Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat.
http://hg.python.org/cpython/rev/0889ab0d0da1

New changeset ef9d42b98a3d by Serhiy Storchaka in branch '2.7':
Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat.
http://hg.python.org/cpython/rev/ef9d42b98a3d

New changeset 6d0fd113a2e4 by Serhiy Storchaka in branch 'default':
Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat.
http://hg.python.org/cpython/rev/6d0fd113a2e4

I'm a little surprised that still exists.

The first version was generated manually.

Wow, I didn't even know that file existed. It could actually help with some static analysis of the C code to verify that the refcounts are as expected w/o actually having to do inter-procedural analysis to figure out all of the refcount values. I wonder if Dave's gcc work could help either verify the file or benefit from it?

As far as I can tell that file is still manually maintained.  I see it mentioned in Doc/conf.py, but nowhere else.  It shouldn't be hard to deal with manually, as the C API doesn't change that often.

A verifier shouldn't be terribly difficult to write, as Python C source is pretty consistently structured.  I'm not volunteering for the task though. <wink>

Were I adding that today, I'd use a more verbose (but more standard)
format, like configparser or JSON.  If any further use is going to be
made of it, that should be considered.  Colon-delimited is a pretty
fragile format.

> Fred L. Drake, Jr. added the comment:
>
> Were I adding that today, I'd use a more verbose (but more standard)
> format, like configparser or JSON.  If any further use is going to be
> made of it, that should be considered.  Colon-delimited is a pretty
> fragile format.

Given the context, it should be okay until Python is rewritten in C++. :-)

I don't know where this is consumed.  Presumably by Sphinx somewhere.

And is it necessary to list all functions there?
Many functions share the same behavior: they don't change the ownership of PyObject* passed as argument, and return a new reference.

Only document functions that don't conform to this rule, like PyTuple_SET_ITEM and PyImport_AddModule.

> And is it necessary to list all functions there?

Probably not.  However, BITD, I was figuring this stuff out as I was
going along by examining the source code and writing down what I
found.  No regard for common or special cases.

S

Since creating refcounts.dat many functions changed their argument's types from `char*` to `const char*`. Here is a patch which fixes mismatches (perhaps not all).

refcounts.dat in 3.x contains PyInt_* functions which don't exist in 3.x.

New changeset ffe24e3e7a2a by Serhiy Storchaka in branch '3.3':
Issue #18085: Add missed const modifier for some entries in refcounts.dat.
http://hg.python.org/cpython/rev/ffe24e3e7a2a

New changeset 6587fd3d89ae by Serhiy Storchaka in branch 'default':
Issue #18085: Add missed const modifier for some entries in refcounts.dat.
http://hg.python.org/cpython/rev/6587fd3d89ae

Can we now close this as a duplicate of issue 9755?

I don't think this is a duplicate of issue 9755; this relates to verifying the data, and that revolves around possible process improvements.

Whether this issue should be closed is tied to whether the file has been verified, as the issue title suggests.

I don't know whether Serhiy verified everything when he made his changes or not; that's not explicit in the issue or commit comments.

No, I hadn't verified all names. I'm sure there are API functions not mentioned in refcounts.dat.

See also issue23903. It would be nice to have a tool that parses Python source headers and produce lists of all public names in different formats for manual updating python3.def, refcounts.dat, docs, etc.

PR 11247 adds more functions and fixes some errors.

New changeset 83dd4e87a62311cfea5fdd37e8a945b6b07bccee by Serhiy Storchaka in branch 'master':
bpo-18085: Update refcounts.dat. (GH-11247)
https://github.com/python/cpython/commit/83dd4e87a62311cfea5fdd37e8a945b6b07bccee

New changeset 73fc14d1f8441aef5ee03be627c63e74a6d915d6 by Miss Islington (bot) in branch '3.7':
bpo-18085: Update refcounts.dat. (GH-11247)
https://github.com/python/cpython/commit/73fc14d1f8441aef5ee03be627c63e74a6d915d6