classification
Title: PyMemberDef missing in limited API / Deprecate structmember.h
Type: enhancement Stage: patch review
Components: C API, Documentation, Interpreter Core Versions: Python 3.10
process
Status: open Resolution:
Dependencies: 24065 28349 Superseder:
Assigned To: belopolsky Nosy List: Arfrever, Matthias Braun, belopolsky, benjamin.peterson, berker.peksag, docs@python, georg.brandl, herzbube, loewis, matrixise, rhettinger, serhiy.storchaka, skip.montanaro, vstinner
Priority: normal Keywords: patch

Created on 2008-05-16 22:45 by benjamin.peterson, last changed 2020-05-27 21:29 by Matthias Braun.

Files
File name Uploaded Description Edit
issue2897.diff belopolsky, 2016-10-03 18:05 review
issue2897-docs-3x.diff belopolsky, 2016-10-05 16:23 review
Pull Requests
URL Status Linked Edit
PR 20462 open Matthias Braun, 2020-05-27 19:52
Messages (21)
msg66972 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-05-16 22:45
As the comment in descrobject.c says:

/* Why is this not included in Python.h? */
msg67028 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-05-18 11:55
We could include it in Py3k.
msg67062 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-05-19 14:43
Note that structmember.h pollutes global namespace with macros that do not  
have conventional Py_ or PY_ prefix.  READONLY and RESTRICTED macros seem 
to be most likely to conflict with other code.  I would be -0 on including tructmember.h in Python.h if flags macros are not properly renamed.  +0 
otherwise.  T_* macros are probably OK, but T prefix is reminiscent of 
popular (in some circles) Taligent naming conventions:

http://pcroot.cern.ch/TaligentDocs/TaligentOnline/DocumentRoot/1.0/Docs/bo
oks/WM/WM_63.html#HEADING77
msg79242 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-06 09:54
Martin, do you want to make the call on this one?
msg79244 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-01-06 10:09
I agree with Alexander; the header shouldn't be included into Python.h
as-is.

I would propose to eliminate it eventually, with the following steps:
1. move PyMemberDef and the function declarations into object.h
2. (simultaneously) introduce properly-prefixed macros in object.h
3. deprecate structmember.h
4. remove it
msg277971 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-10-03 18:05
I am attaching a patch that implements steps 1 and 2 of Martin's plan.  There are over 50 files that include structmember.h. I am not sure it is worth the trouble to update all those files before structmember.h is actually removed.  If we agree that this is the right way forward, I'll make the necessary changes to the docs.
msg277973 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-10-03 18:11
I would also like this opportunity to rename T_PYSSIZET to something more readable: maybe PY_T_PY_SSIZE_T or PY_T_SSIZE_T.
msg277976 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-03 18:20
Please don't forget to use "hg copy" for creating object.h from structmember.h. This preserves the history.

structmember.h should be implemented using object.h. Include object.h and add aliases.

Only READONLY flag is used in 3.x (issue28349). Other flags can be removed.
msg277982 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-10-03 18:33
As explained in #24065, READ_RESTRICTED, PY_WRITE_RESTRICTED and RESTRICTED flags were used for "restricted mode" in Python 2. I don't think we would want to preserve these as we move the rest to object.h.
msg277985 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-03 18:40
PY_T_PY_SSIZE_T is not much readable than PY_T_PYSSIZET. I think it can be just PY_T_SSIZE.
msg277986 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-03 18:41
And please don't miss to fix the documentation in 2.7 and 3.5-3.6.
msg277988 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-10-03 18:50
Changed the title to reflect the way forward and added affected versions to remember to update the documentation.  See #28349 and #24065 for details.
msg278136 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-10-05 16:23
I am attaching a proposed doc patch for Python 3.5+.  For 2.7 we should probably just add a versionchanged note explaining "restricted mode" has been deprecated in Python 2.3.
msg342468 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2019-05-14 14:35
I move this issue to master (3.8)
msg370098 - (view) Author: Matthias Braun (Matthias Braun) * Date: 2020-05-27 15:44
This wasn't mentioned before: Having PyMemberDef part of the structmember.h is a big problem for users of PEP384/limited API, because structmember.h is not part of it.

Which results in the odd situation that `Py_tp_members` or `PyDescr_NewMember()` are part of the limited API but technically you cannot use it because you are not supposed to include headers that are not part of `Python.h`.

The proposed patch here, would fix this!
msg370100 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-27 15:49
> The proposed patch here, would fix this!

The issue title is misleading, it says "Deprecate structmember.h". Is the plan still to deprecate it? Or to make it usable in the limited C API? Please update the title.
msg370101 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-27 15:50
> Note that structmember.h pollutes global namespace with macros that do not have conventional Py_ or PY_ prefix.  READONLY and RESTRICTED macros seem to be most likely to conflict with other code.

One small enhance would be to add such prefix when Py_LIMITED_API is defined.
msg370104 - (view) Author: Matthias Braun (Matthias Braun) * Date: 2020-05-27 15:54
> The issue title is misleading, it says "Deprecate structmember.h". Is the plan still to deprecate it? Or to make it usable in the limited C API? Please update the title.

As far as I understand it: The attached diff, moves the interesting declaration to `object.h` solving the limited API problem. And only leaves structmember.h around for backward compatibility for people using the "old" names `READONLY` or `RESTRICTED`. So in that sense it does deprecate structmember.h

But indeed I hijacked this issue with my complaints about the limited API which may not have been the original intention here, but they get solved nonetheless.
msg370106 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-27 15:59
Also, the bare minimum enhancement would be add rename READONLY to PY_READONLY, but keep a deprecated alias READONLY to PY_READONLY, and update CPython code base to use PY_READONLY. (Same for other similar flags.)
msg370107 - (view) Author: Matthias Braun (Matthias Braun) * Date: 2020-05-27 16:02
Happy to take the proposed diff here (assuming @belopolsky wont mind) and include it into a pull request that also renames the uses of the READONLY flags (and maybe removes the RESTRICTED flags) within cpython source itself.
msg370138 - (view) Author: Matthias Braun (Matthias Braun) * Date: 2020-05-27 21:29
While working on the pull request I felt that the type and constants better fit `descrobject.h` rather than `object.h`.
History
Date User Action Args
2020-05-27 21:29:17Matthias Braunsetmessages: + msg370138
2020-05-27 19:52:48Matthias Braunsetstage: commit review -> patch review
pull_requests: + pull_request19715
2020-05-27 16:02:45Matthias Braunsetmessages: + msg370107
2020-05-27 15:59:28Matthias Braunsettitle: Deprecate structmember.h -> PyMemberDef missing in limited API / Deprecate structmember.h
2020-05-27 15:59:00vstinnersetmessages: + msg370106
2020-05-27 15:54:33Matthias Braunsetmessages: + msg370104
2020-05-27 15:50:25vstinnersetmessages: + msg370101
2020-05-27 15:49:17vstinnersetmessages: + msg370100
2020-05-27 15:46:23Matthias Braunsetcomponents: + C API
versions: + Python 3.10, - Python 3.8
2020-05-27 15:44:39Matthias Braunsetnosy: + Matthias Braun
messages: + msg370098
2019-05-15 02:42:41vstinnerunlinkissue2921 dependencies
2019-05-14 14:35:53matrixisesetnosy: + matrixise, vstinner

messages: + msg342468
versions: + Python 3.8, - Python 2.7, Python 3.5, Python 3.6, Python 3.7
2016-10-10 19:16:00belopolskysetassignee: docs@python -> belopolsky
stage: patch review -> commit review
2016-10-05 16:23:31belopolskysetfiles: + issue2897-docs-3x.diff

messages: + msg278136
2016-10-04 12:05:23berker.peksagsetnosy: + berker.peksag

stage: patch review
2016-10-03 18:50:13belopolskysetassignee: docs@python

components: + Documentation
title: include structmember.h in Python.h -> Deprecate structmember.h
nosy: + docs@python
versions: + Python 2.7, Python 3.5, Python 3.6
messages: + msg277988
2016-10-03 18:43:06belopolskysetdependencies: + Outdated *_RESTRICTED flags in structmember.h, Issues with PyMemberDef flags
2016-10-03 18:41:53serhiy.storchakasetmessages: + msg277986
2016-10-03 18:40:48serhiy.storchakasetmessages: + msg277985
2016-10-03 18:36:03belopolskylinkissue24065 superseder
2016-10-03 18:35:06serhiy.storchakaunlinkissue28349 superseder
2016-10-03 18:33:41belopolskysetmessages: + msg277982
2016-10-03 18:20:17serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg277976
2016-10-03 18:20:01belopolskylinkissue28349 superseder
2016-10-03 18:11:46belopolskysetmessages: + msg277973
2016-10-03 18:05:45belopolskysetfiles: + issue2897.diff
keywords: + patch
messages: + msg277971
2016-10-03 17:33:10belopolskysetnosy: + skip.montanaro

versions: + Python 3.7, - Python 3.4
2012-12-03 08:09:25Arfreversetnosy: + Arfrever
2012-11-30 21:39:09serhiy.storchakasetversions: + Python 3.4, - Python 3.3
2011-03-09 02:05:54terry.reedysetnosy: loewis, georg.brandl, rhettinger, belopolsky, benjamin.peterson, herzbube
versions: + Python 3.3, - Python 3.1
2009-10-17 01:20:04herzbubesetnosy: + herzbube
2009-05-16 22:08:37ajaksu2linkissue2921 dependencies
2009-02-02 15:36:00loewissetassignee: loewis -> (no value)
2009-01-06 10:09:14loewissetmessages: + msg79244
2009-01-06 09:54:18rhettingersetassignee: loewis
messages: + msg79242
nosy: + loewis, rhettinger
2009-01-06 05:02:53gvanrossumsetnosy: - gvanrossum
2009-01-06 05:02:42gvanrossumsetassignee: gvanrossum -> (no value)
versions: + Python 3.1, - Python 3.0
2008-05-19 14:43:55belopolskysetnosy: + belopolsky
messages: + msg67062
2008-05-18 11:55:51georg.brandlsetassignee: gvanrossum
messages: + msg67028
nosy: + georg.brandl, gvanrossum
versions: + Python 3.0, - Python 2.6
2008-05-16 22:45:28benjamin.petersonsetcomponents: + Interpreter Core
versions: + Python 2.6
2008-05-16 22:45:11benjamin.petersoncreate