classification
Title: Add asserts in PyXXX_GET_SIZE macros
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017-03-21 07:53 by serhiy.storchaka, last changed 2017-04-21 23:50 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 751 merged serhiy.storchaka, 2017-03-21 07:55
Messages (5)
msg289927 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-21 07:53
Proposed patch adds asserts for checking the type in macros PyTuple_GET_SIZE, PyList_GET_SIZE and PySet_GET_SIZE. This can help to find the misuse of these macros. Asserts already are used in macros PyBytes_GET_SIZE, PyByteArray_GET_SIZE, PyUnicode_GET_SIZE and PyDict_GET_SIZE. See also the discussion on Python-Dev: https://mail.python.org/pipermail/python-dev/2017-March/147628.html .

This change can break the code that uses these macros for setting the size. For example one place in odictobject.c. But I expect that such cases are rare. And all these macros are not in the limited API.
msg289929 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-03-21 09:54
We should test to run popular C extensions like numpy, PyQt, cython, pillow, etc. with a patched Python to see if it's common to use a "GET" macro to "set" a size. Maybe test also Django see it probably uses many C extensions. I don't ask to test all these modules, but at least a few of them. By "testing", I mean run the test suite of these projects.
msg290637 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-27 19:01
It would be enough to compile them. New macros produce compiler error when used as lvalue.

But I afraid it will take too much time on my netbook.
msg292086 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-21 23:48
New changeset 1a5856bf9295fa73995898d576e0bedf016aee1f by Victor Stinner (Serhiy Storchaka) in branch 'master':
bpo-29867: Add asserts in PyTuple_GET_SIZE, PyList_GET_SIZE and PySet_GET_SIZE. (#751)
https://github.com/python/cpython/commit/1a5856bf9295fa73995898d576e0bedf016aee1f
msg292089 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-21 23:50
I merged your PR. We are at the begining of the 3.7 cycle. Applications can be fixed before 3.7.

In the worst case, we can revert the change.

I like this enhancement, it should help to catch bugs ;-)
History
Date User Action Args
2017-04-21 23:50:52vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg292089

stage: patch review -> resolved
2017-04-21 23:48:13vstinnersetmessages: + msg292086
2017-03-27 19:01:49serhiy.storchakasetmessages: + msg290637
2017-03-21 09:54:48vstinnersetmessages: + msg289929
2017-03-21 07:55:23serhiy.storchakasetpull_requests: + pull_request665
2017-03-21 07:53:03serhiy.storchakacreate