Title: PowerLinux getargs.c FETCH_SIZE endianness bug
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: David.Edelsohn, matejcik, pitrou, python-dev
Priority: normal Keywords:

Created on 2013-05-07 18:29 by David.Edelsohn, last changed 2013-05-16 18:05 by matejcik. This issue is now closed.

Messages (6)
msg188677 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-05-07 18:29
Another endianness bug that causes a failure in

_testcapi reports "string too long" because getargs.c:PyArg_ParseTupleAndKeywords() incorrectly returns a huge value for string_len.

The problem is FETCH_ARGS is passing the wrong type to va_arg.  It grabs an "int" for the size arg, but that is the not the argument type on 64 bit platforms.  This happens to work for little endian because the low part of the 64 bit argument overlaps correctly.  Big endian is not as fortuitous.

If I change "int" to "long", the testcase succeeds.

diff -r a285ce18bd55 Python/getargs.c
--- a/Python/getargs.c	Mon May 06 18:21:10 2013 -0700
+++ b/Python/getargs.c	Tue May 07 11:26:21 2013 -0700
@@ -582,9 +582,9 @@
               char *msgbuf, size_t bufsize, PyObject **freelist)
     /* For # codes */
-#define FETCH_SIZE      int *q=NULL;Py_ssize_t *q2=NULL;\
+#define FETCH_SIZE      long *q=NULL;Py_ssize_t *q2=NULL;\
     if (flags & FLAG_SIZE_T) q2=va_arg(*p_va, Py_ssize_t*); \
-    else q=va_arg(*p_va, int*);
+    else q=va_arg(*p_va, long*);
 #define STORE_SIZE(s)   \
     if (flags & FLAG_SIZE_T) \
         *q2=s; \

I am not certain exactly what type it should be, but it definitely needs to be a matching 64 bit type of 64 bit platforms.

I believe that this bug exists in all versions.
msg188695 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-07 23:57
Is it 2.7-only?
msg188696 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-05-08 00:07
New changeset a199ec80c679 by Antoine Pitrou in branch '2.7':
Issue #17928: Fix test_structmembers on 64-bit big-endian machines.
msg188698 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-08 00:32
Fixed. _testcapi was actually the culprit.
msg189392 - (view) Author: jan matejek (matejcik) * Date: 2013-05-16 17:55
The fix causes regression on my 64bit little-endian machine. It seems that while parsing the arguments, the length value overwrites part of the string pointer.
msg189395 - (view) Author: jan matejek (matejcik) * Date: 2013-05-16 18:05
hmm, but it's caused by a private patch claiming that _testcapimodule.c is PY_SSIZE_T_CLEAN. sorry for the noise.
Date User Action Args
2013-05-16 18:05:04matejciksetmessages: + msg189395
2013-05-16 17:55:31matejciksetnosy: + matejcik
messages: + msg189392
2013-05-08 00:32:38pitrousetstatus: open -> closed
resolution: fixed
messages: + msg188698

stage: resolved
2013-05-08 00:07:22python-devsetnosy: + python-dev
messages: + msg188696
2013-05-07 23:57:29pitrousetmessages: + msg188695
2013-05-07 18:29:22David.Edelsohncreate