This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: PySlice_GetIndicesEx change broke ABI in 3.5 and 3.6 branches
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.6, Python 3.5, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: arigo, benjamin.peterson, cgohlke, cstratak, hroncok, jhelmus, josh.r, larry, mark.dickinson, ned.deily, njs, serhiy.storchaka, skrah, steve.dower
Priority: release blocker Keywords:

Created on 2017-03-29 23:36 by njs, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 1049 merged serhiy.storchaka, 2017-04-08 09:26
PR 1050 merged serhiy.storchaka, 2017-04-08 09:27
PR 1813 merged serhiy.storchaka, 2017-05-25 11:27
PR 2015 dino.viehland, 2017-06-09 01:22
Messages (27)
msg290796 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-03-29 23:36
In the process of fixing issue 27867, a new function PySlice_AdjustIndices was added, and PySlice_GetIndicesEx was converted into a macro that calls this new function. The patch was backported to both the 3.5 and 3.6 branches, was released in 3.6.1, and is currently slated to be released as part of 3.5.4.

Unfortunately, this breaks our normal ABI stability guarantees for micro releases: it means that if a C module that uses PySlice_GetIndicesEx is compiled against e.g. 3.6.1, then it cannot be imported on 3.6.0. This affects a number of high-profile packages (cffi, pandas, mpi4py, dnf, ...). The only workaround is that if you are distributing binary extension modules (e.g. wheels), then you need to be careful not to upgrade to 3.6.1. It's not possible for a wheel to declare that it requires 3.6.1-or-better, because CPython normally follows the rule that we don't make these kinds of changes. Oops.

CC'ing Ned and Larry, because it's possible this should trigger a 3.6.2, and I think it's a blocker for 3.5.4. CC'ing Serhiy as the author of the original patch, since you probably have the best idea how this could be unwound with minimal breakage :-).

python-dev discussion:

Fedora bug:
msg290797 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2017-03-29 23:38
Let's make it a release blocker for now.
msg290806 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-30 05:09
My apologies for breaking the world.

The workaround is to undefine PySlice_GetIndicesEx after #include "Python.h".

#if PY_VERSION_HEX < 0x03070000 && defined(PySlice_GetIndicesEx)
#undef PySlice_GetIndicesEx

But this restores the initial bug.

Other obvious solution -- declare 3.6.0 broken and require upgrading to 3.6.1.

For 3.5.4 we can disable defining the macro when Py_LIMITED_API is not defined. This will restore the initial bug in many extensions. And we need other way to detect if we compile the CPython core or standard extensions for fixing the bug in standard collections. Or expand all uses of PySlice_GetIndicesEx to PySlice_Unpack+PySlice_AdjustIndices (the patch can be generated automatically).
msg290815 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-30 05:59
2.7 is affected too.

Nathaniel, could you add references to this issue and to original issue27867 on the Fedora bug tracker?
msg291118 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2017-04-04 09:29
Currently we haven't updated to Python 3.6.1 at Fedora 26 due to this issue.

While it is a release blocker for 3.6.2, what can be done for 3.6.1?
msg291331 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-08 09:36
PR 1049 and PR 1050 restore ABI compatibility in 3.5 and 2.7. Unfortunately this restores the original bug with using PySlice_GetIndicesEx in third-party code (but CPython core and standard extensions no longer use PySlice_GetIndicesEx).

Can we consider 3.6.0 rather than 3.6.1 as broken release?
msg291406 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-04-10 03:02
> Can we consider 3.6.0 rather than 3.6.1 as broken release?

In the last week, pypi downloads were about evenly split between 3.6.0 and 3.6.1 (2269969 for "3.6.1", 1927189 for "3.6.0", and those two were ~2 orders of magnitude more common than other strings like "3.6.1+", "3.6.0b2", etc. [1]). Not sure what that to conclude from that, but certainly if people start uploading 3.6.1-only wheels right now then it will break things for a lot of end users.

With my manylinux docker image maintainer hat on: we're currently shipping 3.6.0. I'm extremely confident that if we stick with this we'll never get any complaints about the obscure bug with malicious __index__ implementations that's being fixed here. OTOH if we upgrade to 3.6.1, or any version with this ABI change, then we'll definitely get many complaints so long as there's anyone at all still using 3.6.0, which is probably forever. So I'm not sure not sure what incentive we would have to ever upgrade to 3.6.1+ if this ABI change is kept?

(This isn't saying the bug is unimportant! But it sure is hard to sell its importance to folks trying to ship packages and support end-users...)


[1] Somewhat crude query I used in case it's useful for future reference:

  REGEXP_EXTRACT(details.python, r"^([^\.]+\.[^\.]+\.[^\.]+)") as python_version,
  COUNT(*) as download_count,
  REGEXP_MATCH(details.python, r"^3\.6\.")
  download_count DESC
msg291408 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-04-10 03:22
It looks like PyTorch got bitten by this:
msg291743 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-16 07:08
New changeset 89f9eb5b192b875c017d37cac16bd514aad9a801 by Serhiy Storchaka in branch '2.7':
bpo-29943: Remove the PySlice_GetIndicesEx() macro. (#1050)
msg291746 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-16 09:03
New changeset 49a905958ffc2fcd5d1d1a293ae453d45deeb884 by Serhiy Storchaka in branch '3.5':
[3.5] bpo-29943: Do not replace the function PySlice_GetIndicesEx() with a macro (#1049)
msg291979 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-04-20 15:12
msg292276 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-04-25 19:11
Apparently this also broke pyqt for multiple users; here's the maintainers at conda-forge struggling to figure out the best workaround:

I really think it would be good to fix this in 3.6 sooner rather than later. Downstairs projects are accumulating workarounds and pinning 3.6.0 as we speak.
msg292543 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-04-28 17:13
Pillow also had broken wheels up on pypi for a while; they've now put out a bug fix release that #undef's PySlice_GetIndicesEx, basically monkeypatching out the bugfix to get back to the 3.6.0 behavior:
msg292544 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-04-28 18:36
I'm with Nathaniel here: The fixed-bug is probably too obscure to warrant ABI breakage.
msg292715 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-05-02 05:24
More collateral damage: apparently the workaround that Pandas used for this bug (#undef'ing PySlice_GetIndicesEx) broke PyPy, so now they need a workaround for the workaround:

Recording this here partly as a nudge since IIUC the breaking change is still in the 3.6 tree, and partly for the benefit of future projects that are also trying to work around this.
msg292716 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-02 06:18
Either 3.6.1 or 3.6.0 should be officially declared broken. If declare 3.6.1 broken we must port PR 1049 to 3.6 and release or 3.6.2 as early as possible. Many third-party extensions built with 3.6.1 should be rebuild with a new release for compatibility with 3.6.0. Distributors should update Python to a new release. If declare 3.6.0 broken we shouldn't do anything beside this. Distributors should update Python to 3.6.1, as planned.

Ned, what is your decision as RM?
msg292718 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-05-02 06:36
I don't find it helpful to think of it as declaring 3.6.0 broken vs declaring 3.6.1 broken. 3.6.0 is definitely good in the sense that if you build a module on it then it will import on both 3.6.0 and 3.6.1, and 3.6.1 is definitely good in the sense that if you're using it then you can import any module that was built on 3.6.0 or 3.6.1.

I think 3.6.2 should combine the best aspects of 3.6.0 and 3.6.1, in the sense that modules built on 3.6.2 should be importable on any 3.6.x, and 3.6.2 should be able to import modules built on any 3.6.x.

Fortunately this is very easy – it just means that 3.6.2 should continue to export the PySlice_AdjustIndices symbol (because existing 3.6.1 builds will need it to import on 3.6.2), but should remove the #define of PySlice_GetIndicesEx (so code built against 3.6.2 will not *require* PySlice_AdjustIndices to be present).

Removing 3.6.0 and 3.6.1 from circulation is simply not possible; the best 3.6.2 can hope for is to interoperate with them as well as possible.
msg292726 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2017-05-02 08:38
For what it's worth, in Fedora 26 we already rebased Python to 3.6.1, so this issue now is non existent for our ecosystem, and we are not shipping 3.6.0 in any way now.
msg292727 - (view) Author: Miro Hrončok (hroncok) * Date: 2017-05-02 08:46
> For what it's worth, in Fedora 26 we already rebased Python to 3.6.1, so this issue now is non existent for our ecosystem, and we are not shipping 3.6.0 in any way now.

Except of course if 3.6.2 would be fixed in a way that 3.6.1 would be considered broken, and released after Fedora 26 is released. In that case, we would need to deal with the situation in Fedora somehow (e.g. patch it back in or so).
msg293434 - (view) Author: Jonathan Helmus (jhelmus) Date: 2017-05-10 16:41
In Anaconda we ship both Python 3.6.0 and 3.6.1 and have run into this issue when building packages with 3.6.1 and importing them on 3.6.0.  We are discussing adding #undef PySlice_GetIndicesEx to the Python.h we ship with Python 3.6.1.  From the discussion here this seems like a possible solution but we are worried about ABI compatibility with future releases in the 3.6 branch.  Is there a consensus on how this will be addressed in 3.6.2?
msg293452 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-05-10 20:36
@Jonathan: Even 3.6.1 was careful to retain compatibility with code built by 3.6.0. And your proposed 3.6.1-patched will generate binaries equivalent to the ones 3.6.0 generates. So I don't think you need to worry; 3.6.2 is not going to add a new and worse compatibility break, which is what it would take for your proposal to cause a problem. In fact, your 3.6.1-patched is equivalent to what I think 3.6.2 should do. I say go for it; patching 3.6.1 seems like a good solution.
msg294367 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-05-24 17:42
Thanks everyone for the input on this issue.  We've had some discussions here at PyCon US and I think the consensus is that we all agree with Nathaniel's comment above that, for 3.6.x, that modules built on 3.6.2 (and later 3.6.x) should be importable on any 3.6.x (including 3.6.0), and 3.6.2 (and later 3.6.x) should be able to import modules built on any 3.6.x.  Steve Dower has suggested that it would be nice to provide a compile option so that users could compile an extension with the benefit of the 3.6.1 fix if they do not have the need for 3.6.0 compatibility.  I think that would be nice but not essential.  But we do need to ensure that 3.6.2 is fixed to be compatible as above prior to 3.6.2rc currently scheduled for 2017-06-26.  Serhiy, is that doable?

Otherwise, I believe no further changes are needed for the other branches, correct?
msg294390 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-24 20:43
I'll apply to 3.6 the same patch as for 3.5 tomorrow.
msg294474 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-25 12:32
New changeset f43b293f2fee91578e28c7aa566510a0cd6e33cb by Serhiy Storchaka in branch '3.6':
[3.6] bpo-29943: Do not replace the function PySlice_GetIndicesEx() with a macro (GH-1049) (#1813)
msg295481 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-06-09 03:39
As a followup, Nathanial, are you satisfied with the resolution here for the upcoming 3.6.2?
msg295487 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-06-09 04:46
Looks good to me, thanks Serhiy.
msg303101 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2017-09-27 06:28
An update to Serhiy's proposed fix:

#if PY_VERSION_HEX < 0x03070000 && defined(PySlice_GetIndicesEx)
#if !defined(PYPY_VERSION)
#undef PySlice_GetIndicesEx

All PyXxx functions are macros on PyPy, and undefining a macro just makes everything go wrong.  And there is not much PyPy can do about that.
Date User Action Args
2022-04-11 14:58:44adminsetgithub: 74129
2017-09-27 16:19:36josh.rsetnosy: + josh.r
2017-09-27 06:28:16arigosetnosy: + arigo
messages: + msg303101
2017-06-09 04:46:42njssetmessages: + msg295487
2017-06-09 03:39:25ned.deilysetmessages: + msg295481
2017-06-09 01:22:12dino.viehlandsetpull_requests: + pull_request2082
2017-05-25 12:34:20serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-05-25 12:32:11serhiy.storchakasetmessages: + msg294474
2017-05-25 11:27:48serhiy.storchakasetpull_requests: + pull_request1899
2017-05-24 20:43:08serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg294390
2017-05-24 17:42:16ned.deilysetmessages: + msg294367
2017-05-10 20:36:59njssetmessages: + msg293452
2017-05-10 16:41:38jhelmussetnosy: + jhelmus
messages: + msg293434
2017-05-03 08:03:12mark.dickinsonsetnosy: + mark.dickinson
2017-05-02 08:46:10hroncoksetmessages: + msg292727
2017-05-02 08:38:43cstrataksetmessages: + msg292726
2017-05-02 06:36:49njssetmessages: + msg292718
2017-05-02 06:18:27serhiy.storchakasetmessages: + msg292716
2017-05-02 05:24:28njssetmessages: + msg292715
2017-04-28 18:36:14skrahsetnosy: + skrah
messages: + msg292544
2017-04-28 17:13:58njssetmessages: + msg292543
2017-04-25 19:11:23njssetmessages: + msg292276
2017-04-20 15:12:22njssetmessages: + msg291979
2017-04-16 09:03:54serhiy.storchakasetmessages: + msg291746
2017-04-16 07:08:49serhiy.storchakasetmessages: + msg291743
2017-04-10 03:22:55njssetmessages: + msg291408
2017-04-10 03:02:58njssetmessages: + msg291406
2017-04-08 09:36:31serhiy.storchakasettype: behavior
messages: + msg291331
components: + Interpreter Core
stage: patch review
2017-04-08 09:27:37serhiy.storchakasetpull_requests: + pull_request1202
2017-04-08 09:26:33serhiy.storchakasetpull_requests: + pull_request1201
2017-04-06 17:58:21cgohlkesetnosy: + cgohlke
2017-04-04 09:29:28cstrataksetmessages: + msg291118
2017-04-01 09:16:02hroncoksetnosy: + hroncok
2017-03-30 08:56:25cstrataksetnosy: + cstratak
2017-03-30 05:59:17serhiy.storchakasetnosy: + benjamin.peterson

messages: + msg290815
versions: + Python 2.7
2017-03-30 05:09:17serhiy.storchakasetmessages: + msg290806
2017-03-30 01:17:10steve.dowersetnosy: + steve.dower
2017-03-29 23:38:15larrysetpriority: normal -> release blocker

messages: + msg290797
2017-03-29 23:36:28njscreate