classification
Title: Remove "w" format of PyParse_ParseTuple()
Type: Stage:
Components: Interpreter Core Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: haypo, loewis, pitrou
Priority: normal Keywords: patch

Created on 2010-05-29 02:24 by haypo, last changed 2010-06-25 00:03 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
remove_w_format.patch haypo, 2010-06-06 09:03
remove_w_format-2.patch haypo, 2010-06-14 21:37
Messages (8)
msg106702 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-05-29 02:24
"w" format is dangerous because it doesn't give the size of the buffer: the caller may write outside the buffer (buffer overflow).

"w*" and "w#" formats are fine.

It looks like "w" format is not used in trunk nor py3k (only w# and w*).
msg107193 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-06-06 09:03
Attached patch removes "w" format, cleanups the code for "w*" and "w#" formats, and update the documentation.
msg107324 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-06-08 15:09
I'd point out that "w#" is mostly useless too. It's supposed to return a read-write buffer, but as the doc says it also doesn't support "mutable objects", since it isn't able to properly lock/pin the buffer; therefore it probably doesn't support anything useful.
msg107367 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-06-09 00:03
See also #8592: 'y' does not check for embedded NUL bytes.
msg107755 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-06-13 21:36
"w#" is not only useless but also not used in py3k source code. "w" is also not used. Only "w*" is used by fnctl and socket modules.

The problem with w# is that the caller cannot "release" the buffer and so we cannot lock the buffer. I don't know exactly what happens if you get a pointer to a bytearray, release the GIL and then use it, whereas another thread modifies and/or destroys the bytearray object. (I suppose that something bad will happen, like a segfault)
msg107812 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-06-14 21:37
New (improved) version of the patch:
 - Remove also "w#" format
 - Write tests for "w*" format
msg107829 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-06-14 22:39
See also #8926.
msg108572 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-06-25 00:03
Commited to 3.2 (r82208), blocked in 3.1 (r82209).
History
Date User Action Args
2010-06-25 00:03:40hayposetstatus: open -> closed
resolution: fixed
2010-06-25 00:03:26hayposetmessages: + msg108572
2010-06-14 22:39:15hayposetmessages: + msg107829
2010-06-14 21:37:44hayposetfiles: + remove_w_format-2.patch

messages: + msg107812
2010-06-13 21:36:19hayposetmessages: + msg107755
2010-06-09 00:03:06hayposetmessages: + msg107367
2010-06-08 15:09:56pitrousetnosy: + pitrou
messages: + msg107324
2010-06-08 15:08:22pitrousetnosy: + loewis
2010-06-06 09:03:27hayposetfiles: + remove_w_format.patch
keywords: + patch
messages: + msg107193
2010-05-29 02:24:18haypocreate