classification
Title: Exclude the private API from the stable API
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: loewis, python-dev, serhiy.storchaka, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2016-05-01 10:17 by serhiy.storchaka, last changed 2016-09-13 06:29 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
private_limited_api.patch serhiy.storchaka, 2016-05-01 12:04 review
Messages (6)
msg264591 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-01 10:17
Proposed patch excludes the private API from the stable API. This includes:

_Py_Identifier-related API (other _Py_Identifier-related functions are already excluded):

struct _Py_Identifier;
#define _Py_static_string_init(value)
#define _Py_static_string(varname, value)
#define _Py_IDENTIFIER(varname)
PyAPI_FUNC(PyObject*) _PyUnicode_FromId(_Py_Identifier*);
PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, struct _Py_Identifier *);
PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, struct _Py_Identifier *, PyObject *);
PyAPI_FUNC(int) _PyObject_HasAttrId(PyObject *, struct _Py_Identifier *);
PyAPI_FUNC(PyObject *) _PyObject_CallMethodId(PyObject *o, _Py_Identifier *method, const char *format, ...);
PyAPI_FUNC(PyObject *) _PyObject_CallMethodId_SizeT(PyObject *o, _Py_Identifier *name, const char *format, ...);
PyAPI_FUNC(PyObject *) _PyObject_CallMethodIdObjArgs(PyObject *o, struct _Py_Identifier *method, ...);
PyAPI_FUNC(PyObject *) _PyDict_GetItemIdWithError(PyObject *dp, struct _Py_Identifier *key);
PyAPI_FUNC(PyObject *) _PyDict_GetItemId(PyObject *dp, struct _Py_Identifier *key);
PyAPI_FUNC(int) _PyDict_SetItemId(PyObject *dp, struct _Py_Identifier *key, PyObject *item);
PyAPI_FUNC(PyObject *) _PySys_GetObjectId(_Py_Identifier *key);

File helpers that hides OS-specific details and handle EINTR:

PyAPI_FUNC(PyObject *) _Py_device_encoding(int);
PyAPI_FUNC(FILE *) _Py_wfopen(const wchar_t *path, const wchar_t *mode);
PyAPI_FUNC(FILE*) _Py_fopen(const char *pathname, const char *mode);
PyAPI_FUNC(FILE*) _Py_fopen_obj(PyObject *path, const char *mode);
PyAPI_FUNC(Py_ssize_t) _Py_read(int fd, void *buf, size_t count);
PyAPI_FUNC(Py_ssize_t) _Py_write(int fd, const void *buf, size_t count);
PyAPI_FUNC(Py_ssize_t) _Py_write_noraise(int fd, const void *buf, size_t count);
PyAPI_FUNC(int) _Py_wreadlink(const wchar_t *path, wchar_t *buf, size_t bufsiz);
PyAPI_FUNC(wchar_t*) _Py_wrealpath(const wchar_t *path, wchar_t *resolved_path, size_t resolved_path_size);
PyAPI_FUNC(wchar_t*) _Py_wgetcwd(wchar_t *buf, size_t size);

Other OS functions:

PyAPI_FUNC(int) _PyOS_IsMainThread(void);
PyAPI_FUNC(void*) _PyOS_SigintEvent(void);
PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring);
PyAPI_FUNC(int) _PyOS_URandom(void *buffer, Py_ssize_t size);

Old helpers kept for binary compatibility of extensions built before 3.2.4:

PyAPI_FUNC(void) _PyTrash_deposit_object(PyObject*);
PyAPI_FUNC(void) _PyTrash_destroy_chain(void);
PyAPI_DATA(int) _PyTrash_delete_nesting;
PyAPI_DATA(PyObject *) _PyTrash_delete_later;

And other functions and data:

PyAPI_DATA(PyTypeObject) _PyNone_Type;
PyAPI_DATA(PyTypeObject) _PyNotImplemented_Type;
PyAPI_DATA(PyTypeObject) _PyMethodWrapper_Type;
PyAPI_DATA(int) _Py_SwappedOp[];

PyAPI_FUNC(Py_ssize_t) _Py_GetAllocatedBlocks(void);
PyAPI_FUNC(PyObject *) _PyObject_GC_Malloc(size_t size);
PyAPI_FUNC(PyObject *) _PyObject_GC_Calloc(size_t size);
PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *);

PyAPI_FUNC(PyObject*) _Py_strhex(const char* argbuf, const Py_ssize_t arglen);
PyAPI_FUNC(PyObject*) _Py_strhex_bytes(const char* argbuf, const Py_ssize_t arglen);

PyAPI_FUNC(void) _Py_add_one_to_index_F(int nd, Py_ssize_t *index, const Py_ssize_t *shape);
PyAPI_FUNC(void) _Py_add_one_to_index_C(int nd, Py_ssize_t *index, const Py_ssize_t *shape);

PyAPI_DATA(PyTypeObject) _PyNamespace_Type;
PyAPI_FUNC(PyObject *) _PyNamespace_New(PyObject *kwds);

PyAPI_FUNC(int) _PyOS_IsMainThread(void);
PyAPI_FUNC(void*) _PyOS_SigintEvent(void);
PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring);
PyAPI_FUNC(int) _PyOS_URandom(void *buffer, Py_ssize_t size);

PyAPI_FUNC(int) _PyState_AddModule(PyObject*, struct PyModuleDef*);
PyAPI_FUNC(PyThreadState *) _PyThreadState_Prealloc(PyInterpreterState *);
PyAPI_FUNC(void) _PyThreadState_Init(PyThreadState *);
PyAPI_FUNC(void) _PyThreadState_DeleteExcept(PyThreadState *tstate);
PyAPI_FUNC(void) _PyGILState_Reinit(void);
PyAPI_FUNC(PyThreadState *) _PyThreadState_UncheckedGet(void);

PyAPI_FUNC(void) _PyUnicode_ClearStaticStrings(void);
PyAPI_FUNC(int) _PyUnicode_EQ(PyObject *, PyObject *);

PyAPI_FUNC(void) _Py_DumpTraceback(int fd, PyThreadState *tstate);
PyAPI_FUNC(const char*) _Py_DumpTracebackThreads(int fd, PyInterpreterState *interp, PyThreadState *current_tstate);

PyAPI_FUNC(void) _PyImportZip_Init(void);

PyAPI_FUNC(PyObject *) _PyLong_GCD(PyObject *, PyObject *);

All these names are not documented (except _PyObject_GC_Calloc accidentally mentioned in What's New in 3.5) and are not used implicitly in macros (except _PyObject_CallMethodId_SizeT used in _PyObject_CallMethodId).

Many of these functions were added after implementing PEP 384 and changed its signature during its lifetime. They are not explicitly exclude from the stable API only by accident.
msg264594 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-01 12:04
Added

PyAPI_FUNC(PyObject *) _PyObject_GenericGetAttrWithDict(PyObject *, PyObject *, PyObject *);
PyAPI_FUNC(int) _PyObject_GenericSetAttrWithDict(PyObject *, PyObject *, PyObject *, PyObject *);
PyAPI_FUNC(int) _Py_fstat(int fd, struct _Py_stat_struct *status);
PyAPI_FUNC(void) _PyEval_FiniThreads(void);
msg264613 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-05-02 02:41
#23903 is closely related, but I haven't made it back to that one in some time.
msg275741 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-11 08:03
New changeset d00f15af75ea by Serhiy Storchaka in branch 'default':
Issue #26900: Excluded underscored names and other private API from limited API.
https://hg.python.org/cpython/rev/d00f15af75ea
msg275746 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-11 08:39
I didn't follow this discussion, but thanks for the fix Serhiy :-) By
the way, I started a thread on python-dev to discuss a new
organization of the C API for Python 3.7.
msg275747 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-11 08:50
Maybe new private functions was added after writing the patch.

And many new public functions was added without guarding by specific Py_LIMITED_API value. This is separate issue.
History
Date User Action Args
2016-09-13 06:29:27serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2016-09-11 08:50:48serhiy.storchakasetmessages: + msg275747
2016-09-11 08:39:10vstinnersetmessages: + msg275746
2016-09-11 08:03:44python-devsetnosy: + python-dev
messages: + msg275741
2016-09-10 17:14:04serhiy.storchakasetassignee: serhiy.storchaka
2016-05-02 02:41:28zach.waresetnosy: + zach.ware
messages: + msg264613
2016-05-01 12:04:47serhiy.storchakasetfiles: - private_limited_api.diff
2016-05-01 12:04:37serhiy.storchakasetfiles: + private_limited_api.patch

messages: + msg264594
2016-05-01 10:17:53serhiy.storchakacreate