classification
Title: PyModuleDef_HEAD_INIT does not explicitly initialize all fields of m_base
Type: compile error Stage: resolved
Components: Extension Modules Versions: Python 3.1, Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, dmalcolm, loewis, pitrou, scoder
Priority: normal Keywords: patch

Created on 2010-08-04 23:43 by dmalcolm, last changed 2010-12-08 22:40 by dmalcolm. This issue is now closed.

Files
File name Uploaded Description Edit
py3k-initialize-all-of-m_base.patch dmalcolm, 2010-08-04 23:43 review
Messages (7)
msg112926 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-08-04 23:43
Attempting to compile Python 3 extension modules on GCC with "-Wmissing-field-initializers" enabled leads to warnings from the PyModuleDef_HEAD_INIT macro

Seen attempting to build SELinux python bindings against python 3.1 with "-W -Werror", the "-W" implies "-Wmissing-field-initializers":
> cc -Werror -Wall -W -Wundef -Wshadow -Wmissing-noreturn
> > -Wmissing-format-attribute -I../include -I/usr/include -D_GNU_SOURCE
> > -D_FILE_OFFSET_BITS=64  -I/usr/include/python3.1 -fPIC -DSHARED -c -o
> > audit2why.lo audit2why.c
> > cc1: warnings being treated as errors
> > audit2why.c:439: error: missing initializer
> > audit2why.c:439: error: (near initialization for ?moduledef.m_base.m_init¹)
> > make: *** [audit2why.lo] Error 1

The PyModuleDef_HEAD_INIT is intended to initialize m_base within a PyModuleDef, but only explicitly initializes the PyObject_HEAD fields:

#define PyModuleDef_HEAD_INIT {PyObject_HEAD_INIT(NULL)}

typedef struct PyModuleDef_Base {
  PyObject_HEAD
  PyObject* (*m_init)(void);
  Py_ssize_t m_index;
  PyObject* m_copy;
} PyModuleDef_Base;

typedef struct PyModuleDef{
  PyModuleDef_Base m_base;
  const char* m_name;
  const char* m_doc;
  Py_ssize_t m_size;
  PyMethodDef *m_methods;
  inquiry m_reload;
  traverseproc m_traverse;
  inquiry m_clear;
  freefunc m_free;
} PyModuleDef;

The attached patch extends it to also explicitly zero the other m_base fields.
msg117553 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-09-28 21:36
the patch looks OK, but out of curiosity: do you really declare all the fields of a PyTypeObject?
This structure is really designed so that newer members are left at the end; most types don't need to initialize them, C standard ensures that they will be zero.
msg117560 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-09-28 22:46
Thanks.  The code in question is a wrapper to a security-sensitive library (user-space SELinux code), hence the compilation warnings have been turned up as much as possible.

The .c code in question is generated by SWIG, and that does indeed appear to be writing out full initializers for PyTypeObject instances (and the other associated structs).

It appears to be just Python 3's PyModuleDef_HEAD_INIT macro that leaves fields uninitialized (hence this patch).

The gory details of the SWIG-generated code can be seen at:
  http://userspace.selinuxproject.org/trac/browser/libselinux/src/selinuxswig_wrap.c
(and the .i files in that directory)

Although it's not on by default gcc will issue a "missing initializer" warning when fields aren't initialized when "-Wmissing-field-initializers" is enabled (in this case, due to the use of "-W").  This becomes an error with -Werror.

Whether or not this is a useful warning isn't clear to me, but it seems to be reasonable to suppress the warning given that as-is, people who use gcc's "-W" catch-all will run into this on all Python 3 modules, and that the patch is trivial, and that this case gives no warnings when building such code against Python 2.*

Hope this makes sense.
msg121372 - (view) Author: Stefan Behnel (scoder) * Date: 2010-11-17 18:17
I agree that this is annoying, we get the same thing in Cython's test suite all over the place. Any foreign warning that doesn't get triggered helps in debugging your own code. And this one is easy to avoid.
msg121382 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-17 20:42
This looks ok to me.
msg121385 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-11-17 21:20
Fix committed to py3k as r86499
msg123658 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-12-08 22:40
Forgot to close this one out
History
Date User Action Args
2013-01-27 10:57:19serhiy.storchakalinkissue7576 superseder
2010-12-08 22:40:49dmalcolmsetstage: patch review -> resolved
2010-12-08 22:40:27dmalcolmsetstatus: open -> closed
resolution: fixed
messages: + msg123658
2010-11-17 21:20:41dmalcolmsetmessages: + msg121385
2010-11-17 20:42:10pitrousetnosy: + pitrou
messages: + msg121382
2010-11-17 18:17:29scodersetnosy: + scoder
messages: + msg121372
2010-09-28 22:46:43dmalcolmsetmessages: + msg117560
2010-09-28 21:36:23amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg117553
2010-08-05 12:52:54pitrousetkeywords: patch, patch
nosy: + loewis

versions: - Python 3.3
2010-08-04 23:43:54dmalcolmsetkeywords: patch, patch
title: PyModuleDef_HEAD_INIT does not explicitly initial all fields of m_base -> PyModuleDef_HEAD_INIT does not explicitly initialize all fields of m_base
2010-08-04 23:43:31dmalcolmcreate