classification
Title: Use GCC __attribute__((alloc_size(x, y))) on PyMem_Malloc() functions
Type: Stage:
Components: Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: haypo, svelankar
Priority: normal Keywords:

Created on 2017-02-28 08:54 by haypo, last changed 2017-03-15 01:36 by svelankar.

Pull Requests
URL Status Linked Edit
PR 669 closed svelankar, 2017-03-15 01:36
Messages (5)
msg288699 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-28 08:54
GCC allows to get "size" parameters of functions allocating memory to emit better warning. For example, GCC 7 will detect implicit cast from signed to unsigned integer and emit a warning.

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html

Example of Python functions that can benefit of this attribute:

* PyMem_RawMalloc(), PyMem_RawCalloc(), PyMem_RawRealloc()
* PyMem_Malloc(), PyMem_Calloc(), PyMem_Realloc()
* PyObject_Malloc(), PyObject_Calloc(), PyObject_Realloc()
msg289308 - (view) Author: svelankar (svelankar) * Date: 2017-03-09 17:08
So once these functions are decorated with this attribute, what kind of testing/validation you have in mind, please let me know.
msg289310 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-03-09 17:13
> So once these functions are decorated with this attribute, what kind of testing/validation you have in mind, please let me know.

Call PyMem_Malloc(Py_ssize_t) for example: it must emit a warning on GCC 7, since casting negative values to size_t overflows.

Not sure how to test the attribute on GCC 6. Maybe some GCC related static analyzers are able to detect memory leaks like:

void test(void)
{
   void *ptr = PyMem_Malloc(16);
   /* don't free ptr */
}
msg289612 - (view) Author: svelankar (svelankar) * Date: 2017-03-14 17:22
Ok. 

As a side note, while compiling python source using gcc 7 [gcc (GCC) 7.0.1 20170314 (experimental)], few places in the code with case fallthrough (must be intentional) triggered this warning - -Wimplicit-fallthrough=.

We can either disable this warning altogether (downside being unintended fallthroughs will go unnoticed) OR  pass some flag [https://developers.redhat.com/blog/2017/03/10/wimplicit-fallthrough-in-gcc-7/] to -Wimplicit-fallthrough=. so that it does a regex match on the comments defined [something like  /* fall through code */] in that specific part of the code and suppresses the warning. The downside to this is that these comments might have to be inserted wherever they are missing and new code introduced in the future with intentional fallthroughs need to write those comments. Please let me know.
msg289616 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-03-14 20:32
Yeah, as for each GCC release, I expect new warnings. I noticed the
implicit fall through in GCC 7. I know that it's used on purpose in CPython.
History
Date User Action Args
2017-03-15 01:36:42svelankarsetpull_requests: + pull_request552
2017-03-14 20:32:15hayposetmessages: + msg289616
title: Use GCC __attribute__((alloc_size(x,y))) on PyMem_Malloc() functions -> Use GCC __attribute__((alloc_size(x, y))) on PyMem_Malloc() functions
2017-03-14 17:22:46svelankarsetmessages: + msg289612
2017-03-09 17:13:17hayposetmessages: + msg289310
2017-03-09 17:08:06svelankarsetnosy: + svelankar
messages: + msg289308
2017-02-28 08:54:10haypocreate