Message105419
Here is a new patch.
Usage, short version:
By default, modules compile in Unicode-agnostic mode, where Py_UNICODE is an incomplete type. Unicode-agnostic modules will load in both UCS2 and UCS4 interpreters.
If a module defines PY_REAL_PY_UNICODE before including Python.h, Py_UNICODE will be a complete type. The module can then dereference Py_UNICODE pointers and use sizeof(Py_UNICODE). Attempting to load the module into a mismatched interpreter will cause an ImportError.
Longer version:
In Unicode-agnostic mode, extensions can pass around Py_UNICODE pointers freely, but they will get a compilation error if they attempt to dereference the pointer or use sizeof(Py_UNICODE). In Unicode-agnostic mode, the following are never defined: Py_UNICODE_SIZE, Py_UNICODE_WIDE, HAVE_USABLE_CHAR_T, and functions and macros take Py_UNICODE as parameter or return it.
Passing a Py_UNICODE pointer to a function that expects a wchar_t * (or something similar) will compile, but will generate an incompatible pointer types warning.
Per MvL's suggestion, the patch adds a flag field to PyModuleDef_Base and defines two flags: one to indicate if the module depends on the Unicode width, and another to indicate the width the module was compiled with. These flags are checked in PyModule_Create2.
The patch includes unit tests to ensure that importing works or doesn't work in the correct cases.
19 of the 77 modules in Modules/ needed PY_REAL_PY_UNICODE.
18 of them failed to compile without it (good!).
1 of them (_tkinter.c) output an "incompatible pointer types" warning without it, but compiled anyway (resulting in bad code - without evidence to the contrary it assumed Python was UCS2).
I ran the test suite with a UCS2 Python and a UCS4 Python on Linux. I had hoped to run the test suite on Windows as well, but many tests are failing on Windows even without the patch. |
|
Date |
User |
Action |
Args |
2010-05-09 22:14:51 | stutzbach | set | recipients:
+ stutzbach, lemburg, gvanrossum, loewis, zooko, scoder, vstinner, Arfrever, r.david.murray |
2010-05-09 22:14:51 | stutzbach | set | messageid: <1273443291.26.0.304983963221.issue8654@psf.upfronthosting.co.za> |
2010-05-09 22:14:49 | stutzbach | link | issue8654 messages |
2010-05-09 22:14:48 | stutzbach | create | |
|