classification
Title: Backwards compatibility support for Py_ssize_t
Type: enhancement Stage: test needed
Components: Extension Modules Versions: Python 3.2
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: hinsen, loewis, pitrou
Priority: normal Keywords:

Created on 2006-05-10 13:28 by hinsen, last changed 2010-08-09 16:37 by pitrou. This issue is now closed.

Messages (7)
msg61231 - (view) Author: Konrad Hinsen (hinsen) Date: 2006-05-10 13:28
PEP353 recommends to add the following code snippet to extension 
modules to ensure compatibility with pre-2.5 interpreters:

#if PY_VERSION_HEX < 0x02050000
typedef int Py_ssize_t;
#define PY_SSIZE_T_MAX INT_MAX
#define PY_SSIZE_T_MIN INT_MIN
#endif

This is insufficient, because type definitions that use Py_ssize_t must 
also be added. Here is what works for me, though they may be more 
definitions to be included:

#if PY_VERSION_HEX < 0x02050000
typedef int Py_ssize_t;
#define PY_SSIZE_T_MAX INT_MAX
#define PY_SSIZE_T_MIN INT_MIN
typedef Py_ssize_t (*lenfunc)(PyObject *);
typedef PyObject *(*ssizeargfunc)(PyObject *, Py_ssize_t);
typedef PyObject *(*ssizessizeargfunc)(PyObject *, Py_ssize_t, 
Py_ssize_t);
typedef int(*ssizeobjargproc)(PyObject *, Py_ssize_t, PyObject *);
typedef int(*ssizessizeobjargproc)(PyObject *, Py_ssize_t, Py_ssize_t, 
PyObject *);
typedef Py_ssize_t (*readbufferproc)(PyObject *, Py_ssize_t, void **);
typedef Py_ssize_t (*writebufferproc)(PyObject *, Py_ssize_t, void **);
typedef Py_ssize_t (*segcountproc)(PyObject *, Py_ssize_t *);
typedef Py_ssize_t (*charbufferproc)(PyObject *, Py_ssize_t, char **);
#endif

However, the main problem is elsewhere. Extension modules may well 
need to use Py_ssize_t in their own data types, and may furthermore 
with to make these data types available to yet other extension 
modules. In practice, this requires the inclusion of a code block such 
as the one shown above inside header files. However, this requires a 
mechanism for avoiding redefinitions, which at the moment does not 
seem to exist. My request is to add such a mechanism to Python 2.5 
and recommend its consistent use in an update of PEP353.

Concretely, I propose that Python 2.5 should define a macro 
PY_HAS_PY_SSIZE_T, and that PEP353 should recommend the inclusion 
of the code snippet

#ifndef PY_HAS_PY_SSIZE_T
#define PY_HAS_PY_SSIZE_T
typedef int Py_ssize_t;

/* add all the other definitions here */

#endif

that contains an exhaustive set of definitions that depend on 
Py_ssize_t.
msg61232 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-05-12 14:06
Logged In: YES 
user_id=21627

Please re-read the section immediately following the #ifdef
code in PEP 353. It explains how you should avoid these
other typedefs.
msg61233 - (view) Author: Konrad Hinsen (hinsen) Date: 2006-05-12 14:18
Logged In: YES 
user_id=11850

I have read that section, but I am not yet convinced that I can avoid those 
casts with all common C compilers - and since I cannot try them all, I'd rather 
play safe and keep the casts. If they were never necessary, why were those 
typedefs introduced at all?

Anyway, it is not the additional typedefs that are the essence of my feature 
request. The main issue is that if multiple header files introduce Py_ssize_t, 
the compiler will stop with an error message.
msg61234 - (view) Author: Konrad Hinsen (hinsen) Date: 2006-05-12 14:26
Logged In: YES 
user_id=11850

Here is an illustration of my problem. Given the following three files:

-- foo.h -------------------------
#include <Python.h>

#if PY_VERSION_HEX < 0x02050000
typedef int Py_ssize_t;
#endif

typedef struct {
  Py_ssize_t index;
} foo;


-- bar.h -------------------------

#include <Python.h>

#if PY_VERSION_HEX < 0x02050000
typedef int Py_ssize_t;
#endif

typedef struct {
  Py_ssize_t index;
} bar;


-- foobar.c ----------------------

#include "foo.h"
#include "bar.h"

foo a;
bar b;

int main(int argc, char **argv) {
  return 0;
}

----------------------------------

I get from gcc:

gcc foobar.c 
In file included from foobar.c:1:
foo.h:1:20: error: Python.h: No such file or directory
In file included from foobar.c:2:
bar.h:4: error: redefinition of typedef 'Py_ssize_t'
foo.h:4: error: previous declaration of 'Py_ssize_t' was here

I see no solution to this problem that would work in the most general case in 
which all three files are part of different packages written by different 
authors, i.e. in the absence of a coordination.
msg61235 - (view) Author: Konrad Hinsen (hinsen) Date: 2006-05-12 14:34
Logged In: YES 
user_id=11850

Oops, I forgot the -I option, but that doesn't really make a difference.

For Python 2.5:

gcc -I /usr/local/include/python2.5 foobar.c 

--> no error message

For Python 2.4:

gcc -I /Library/Frameworks/Python.framework/Versions/2.4/include/
python2.4 foobar.c 
In file included from /Library/Frameworks/Python.framework/Versions/2.4/
include/python2.4/Python.h:55,
                 from foo.h:1,
                 from foobar.c:1:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/
pyport.h:396: warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/
pyport.h:397: warning: 'struct winsize' declared inside parameter list
In file included from foobar.c:2:
bar.h:4: error: redefinition of typedef 'Py_ssize_t'
foo.h:4: error: previous declaration of 'Py_ssize_t' was here
msg61236 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-05-12 20:32
Logged In: YES 
user_id=21627

I understand your request, and I sympathize with it
(although I would call the macro Py_ssize_t_defined).

However, I find it equally important that the other issue
gets understood, as well. The macros are not necessary for
portable code, and they never were.

They were introduced for convenience only, so that you can
have the actual type name for the self parameter (e.g.
FooObject* instead of PyObject*). If the signatures are
corrected to have PyObject* as their first parameter, the
casts *should* become unnecessary. If they are then still
required, that indicates a serious programming error.

The evilness of these casts comes from the fact that they
can silence warnings that would point to severe type errors
if the cast wouldn't silence them. For example, if the
parameter order or the number is wrong for one of these
functions, the compiler won't notice because of the cast.
The cast is only there to convert the first parameter
(self), yet it can manage to convert any other parameter, as
well. 

So getting these function pointers type correct not only
increases portabiltiy in the presence of Py_ssize_t, but
also improves correctness and readability of the code.
msg113426 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-09 16:37
This request, however legitimate, is completely outdated.
History
Date User Action Args
2010-08-09 16:37:59pitrousetstatus: open -> closed

nosy: + pitrou
messages: + msg113426

resolution: out of date
2010-08-09 03:24:50terry.reedysetversions: + Python 3.2, - Python 3.1, Python 2.7
2009-03-21 02:44:58ajaksu2setstage: test needed
versions: + Python 3.1, Python 2.7
2006-05-10 13:28:03hinsencreate