This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Backport capsule object
Type: enhancement Stage: needs patch
Components: Interpreter Core Versions: Python 2.7
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: larry Nosy List: benjamin.peterson, brian.curtin, flox, jcea, larry, lemburg, pitrou, vstinner
Priority: normal Keywords: buildbot, patch

Created on 2010-02-22 18:28 by larry, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (15)
msg99802 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2010-02-22 18:28
Backporting the PyCapsule object from 3.1 to 2.7.  I'll have a patch today.
msg101691 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-03-25 12:25
All windows buildbots fail to compile:
msg101695 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-03-25 13:31
As Marc-André pointed out, you shouldn't have converted old uses of PyCObject. It breaks compatibility, and I don't think there's any point since 2.7 will probably be the last in the 2.x line.
msg101696 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-03-25 13:37
(if you want to be perfectionist, you can have those modules provide both a PyCObject and a PyCapsule interface...)
msg101698 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-03-25 14:09
PyCapsule comes from issues #5630 and #5872.
msg101723 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2010-03-25 17:41
This is also (naturally) being discussed on python-dev.  I've posted a long message there about it; you can find that message here:

I don't know whether it's best to conduct such a discussion here, in Python-Dev, or both.  I assumed Python-Dev is superior because more people will see it.  If it makes sense to move the discussion here we can move it.
msg101726 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2010-03-25 18:34
Florent: that's because the Windows build projects don't build capsule.c.  I don't have an appropriate Windows development environment; I'll try to fix them by hand tonight, but if that doesn't work we'll have to call for help from some Windows core developer.
msg101727 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-03-25 18:38
(adding myself to the nosy list as a Windows dev)
msg102164 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-04-02 12:51
The PendingDeprecationWarning is still spit by the bsddb module.

$ ./python -Wd -m test.regrtest test_bsddb
./Lib/importlib/ PendingDeprecationWarning: The CObject type is marked Pending Deprecation in Python 2.7.  Please use capsule objects instead.
1 test OK.
msg102192 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2010-04-02 20:14
Yes.  I was told it was inappropriate for me to change the bsddb module, as it's independently maintained by Jesus Cea.  I sent Mr. Cea a patch last night; I hope he chooses to merge it soon.

Since CObjects are marked Pending Deprecation, and bsddb is using a CObject, this is correct behavior.

Were you mentioning it simply to bring it my attention, or am I missing something important?
msg102194 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2010-04-02 20:19
Oh, and, last night I checked in r79590.  This is the checkin that ameliorates the backwards compatibility issues.  Specifically, I changed four things:

* PyCObject_AsVoidPtr() can now open capsules.  This addresses
  most of the remaining backwards-compatibility concerns about
  the conversion of Python 2.7 from CObjects to capsules.
* CObjects were marked Pending Deprecation.
* Documentation about this pending deprecation was added to
* The capsule source files were added to the legacy PC build

(Someone had already added them to the current PC build process, a day or two after I committed the previous checkin.)

Sorry for forgetting to update the issue last night, but I was in a bit of a hurry to get this done before Benjamin cut 2.7b1.
msg102196 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-04-02 20:25
Ok, I put Jésus in the nosy list.

It makes the test___all__ fail on trunk because it checks the warnings raised during import.

AFAIU, the patch is something like:

-#if (PY_VERSION_HEX < 0x03020000)
+#if (PY_VERSION_HEX < 0x02070000)
msg102203 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2010-04-02 21:06
The patch is a bit more involved than that.  Capsules didn't exist in 3.0, and the bsddb module published a CObject in 3.1.  So bsddb must continue to use CObject for those two releases.

Therefore the patch to the line you were addressing looks like this:

-#if (PY_VERSION_HEX < 0x03020000)
+#if (PY_VERSION_HEX < ((PY_MAJOR_VERSION < 3) ? 0x02070000 : 0x03020000))

However, Jesus's use of capsules has a bug.  The capsule API requires that the string passed in to PyCapsule_New() must "outlive" the capsule; he's passing in a stack buffer.  So my patch also fixes that.

Finally my bsddb patch adds the appropriate text to the header file where it describes how to import the _bsddb C API object (CObject vs capsule).

FWIW, I want to close this issue soon.  How about I close it after 2.7b1 has been out for a week or so--assuming there aren't any new concerns regarding the capsule backport.
msg103661 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2010-04-20 00:04
Marking closed, as promised; 2.7b1 has been out for nine days and there hasn't been a peep.  If new problems crop up from the capsule backport, please create a new issue and assign it to me.

For posterity: 2.7b1 shipped with a bsddb that still publishes a CObject.  This is the only CObject left in 2.7b1, and Python doesn't itself use that CObject for anything.  Therefore there are no CObject exploits left in pure Python 2.7b1.  I don't know if bsddb will switch to a capsule in a later Python 2.7 beta; that's up to Jesus and Benjamin and co.  But I suspect it won't.
msg104140 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2010-04-25 03:07
For future reference:

The final consensus was not to integrate bsddb Capsule for Python 2.7, since CObject is not actually deprecated in python 2.7.
Date User Action Args
2022-04-11 14:56:58adminsetgithub: 52240
2010-04-25 03:07:57jceasetmessages: + msg104140
2010-04-20 00:04:42larrysetstatus: open -> closed
resolution: accepted
messages: + msg103661
2010-04-02 21:06:16larrysetmessages: + msg102203
2010-04-02 20:25:28floxsetnosy: + jcea
messages: + msg102196
2010-04-02 20:19:21larrysetmessages: + msg102194
2010-04-02 20:14:58larrysetmessages: + msg102192
2010-04-02 12:51:30floxsetmessages: + msg102164
2010-03-25 18:38:39brian.curtinsetnosy: + brian.curtin
messages: + msg101727
2010-03-25 18:34:36larrysetmessages: + msg101726
2010-03-25 17:41:26larrysetmessages: + msg101723
2010-03-25 14:09:56vstinnersetnosy: + vstinner
messages: + msg101698
2010-03-25 13:37:15pitrousetmessages: + msg101696
2010-03-25 13:31:27pitrousetnosy: + pitrou, lemburg
messages: + msg101695
2010-03-25 12:25:39floxsetkeywords: + buildbot
nosy: + flox
messages: + msg101691

2010-02-22 18:28:45larrycreate