This function is incredibly useful for efficient interoperability between Python and other languages with UTF-8 based strings (e.g. Rust). Right now it's not possible to do interop without several copies/allocations if you're trying to build an abi3 wheel. This is tactically relevant to me here: https://github.com/PyO3/pyo3/issues/1125

This API has been stable since it was introduced in Python 3.1, so I think making it stable would be appropriate.

Inada, Benjamin suggested I should ask you for your feedback on this. I'm happy to send a pull request.

+1. It is a very important API.

What about PyUnicode_AsUTF8? Should it be made public too or left for internal use only?

What about third-party implementations of Python? How hard to implement this API on an implementation without reference counts? It is interesting to hear the expert opinion of the core developers of PyPy.

I think less is more, one API is plenty :-)

It looks to me like the API is already supported on PyPy, so I think it's fine from that perspective: https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/unicodeobject.py#L493

PyUnicode_AsUTF8() is used 3 times more than PyUnicode_AsUTF8AndSize().

$ find -type f -name '*.c' -exec egrep 'PyUnicode_AsUTF8AndSize\(' '{}' + | wc -l
35
$ find -type f -name '*.c' -exec egrep 'PyUnicode_AsUTF8\(' '{}' + | wc -l
101

PyUnicode_AsUTF8 is useful "API". But it can be implemented as C macro, C inline function, or functions/macros in any other languages using PyUnicode_AsUTF8AndSize.

PyUnicode_AsUTF8AndSize is more important for "ABI".

I agree about PyUnicode_AsUTF8.

But I think it would be worth to ask PyPy team about PyUnicode_AsUTF8AndSize.

An alternate C API is PyUnicode_GetUTF8Buffer (issue39087). It requires explicit releasing the buffer after use, so it can be used even on implementations with moving garbage collector.

Py_buffer is not part of the limited API at all, so I don't think it's usable for this.

Oh, would not be worth to add Py_buffer to the limited API?

It's a big project I think :-) Py_Buffer is allocated on the stack, so either we'd have to agree to never change it's ABI (size, alignment, etc.) or we'd need to completely change the interface.

Agreed that there's no way we can make Py_buffer part of the limited ABI.

I just looked over the PR and it's missing a What's New entry (e.g. https://docs.python.org/3/whatsnew/3.9.html#c-api-changes). Other than that, looks fine to me.

New changeset 3a8fdb28794b2f19f6c8464378fb8b46bce1f5f4 by Alex Gaynor in branch 'master':
bpo-41784: make PyUnicode_AsUTF8AndSize part of the limited API (GH-22252)
https://github.com/python/cpython/commit/3a8fdb28794b2f19f6c8464378fb8b46bce1f5f4

Thanks, Alex!