This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eric.snow
Recipients eric.snow, serhiy.storchaka, vstinner
Date 2022-01-26.22:01:04
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1643234464.46.0.949364478641.issue46541@roundup.psfhosted.org>
In-reply-to
Content
`_Py_Identifier` has been useful but at this point there is a faster and simpler approach we could take as a replacement: statically initialize the objects as fields on `_PyRuntimeState` and reference them directly through a macro.

This would involve the following:

* add a `PyUnicodeObject field (not a pointer) to `_PyRuntimeState` for each string that currently uses `_Py_IDENTIFIER()`
* initialize each object as part of the static initializer for `_PyRuntimeState`
* make each "immortal" (e.g. start with a really high refcount)
* add a macro to look up a given string
* update each location that currently uses `_Py_IDENTIFIER()` to use the new macro instead

As part of this, we would also do the following:

* get rid of all C-API functions with `_Py_Identifer` parameters
* get rid of the old runtime state related to identifiers
* get rid of `_Py_Identifier`, `_Py_IDENTIFIER()`, etc.

(Note that there are several hundred uses of `_Py_IDENTIFIER()`, including a number of duplicates.)


Pros:

* reduces indirection (and extra calls) for C-API using the strings (making the code easier to understand and speeding it up)
* the objects are referenced from a fixed address in the static data section (speeding things up and allowing the C compiler to optimize better)
* there is no lazy allocation (or lookup, etc.) so there are fewer possible failures when the objects get used (thus less error return checking)
* simplifies the runtime state
* saves memory (at little, at least)
* the approach for per-interpreter is simpler (if needed)
* reduces the number of static variables in any given C module
* reduces the number of functions in the ("private") C-API
* "deep frozen" modules can use these strings
* other commonly-used strings could be pre-allocated by adding `_PyRuntimeState` fields for them

Cons:

* churn
* adding a string to the list requires modifying a separate file from the one where you actually want to use the string
* strings can get "orphaned" (we could prevent this with a check in `make check`)
* some PyPI packages may rely on `_Py_IDENTIFIER()` (even though it is "private" C-API)
* some strings may never get used for any given ./python invocation


Note that with a basic partial implementation (GH-30928) I'm seeing a 1% improvement in performance (see https://github.com/faster-cpython/ideas/issues/230).
History
Date User Action Args
2022-01-26 22:01:04eric.snowsetrecipients: + eric.snow, vstinner, serhiy.storchaka
2022-01-26 22:01:04eric.snowsetmessageid: <1643234464.46.0.949364478641.issue46541@roundup.psfhosted.org>
2022-01-26 22:01:04eric.snowlinkissue46541 messages
2022-01-26 22:01:04eric.snowcreate