classification
Title: Dangerous usage of "O" format string in _asynciomodule.c
Type: Stage:
Components: asyncio Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, python-dev, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2016-12-09 13:38 by vstinner, last changed 2016-12-15 08:10 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
_asyncio.patch vstinner, 2016-12-09 13:45 review
Messages (5)
msg282778 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-12-09 13:38
The new _asyncio module of Python 3.6 uses the _PyObject_CallMethodId() function to call functions. This function has a weird behaviour when using the format string "O": if the object is a tuple, the tuple is unpacked.

_PyObject_CallMethodId(obj, &PyId_meth, "O", tuple, NULL) calls obj.meth(*tuple) instead of obj.meth(tuple).

I only found one function which may have the bug: task_call_step(). But it seems like this function cannot be called with a tuple as "arg", only with an exception object.

But just in case, I would suggest to replace:
   _PyObject_CallMethodId(obj, nameid, "O", arg);
with
   _PyObject_CallMethodIdObjArgs(obj, nameid, arg, NULL);

Note: _PyObject_CallMethodId() is called with a NULL terminal in the argument list, but the NULL is useless. A terminator is only required by _PyObject_CallMethodIdObjArgs(). Yeah, Python has a wide choice of functions to call a callable object, with funny APIs... And I'm adding new ones to Python 3.7 ;-)
msg282779 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-12-09 13:38
Previous similar bug in generators: issue #21209.
msg282781 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-12-09 13:44
The issue was in fixed in Python 3.7 with a change made for optimization, not directly to fix this issue:

changeset:   105547:b29c719d5992
tag:         tip
user:        Victor Stinner <victor.stinner@gmail.com>
date:        Fri Dec 09 14:24:02 2016 +0100
files:       Modules/_asynciomodule.c
description:
Use _PyObject_CallMethodIdObjArgs() in _asyncio

Issue #28915: Replace _PyObject_CallMethodId() with
_PyObject_CallMethodIdObjArgs() when the format string was only made of "O"
formats, PyObject* arguments.

_PyObject_CallMethodIdObjArgs() avoids the creation of a temporary tuple and
doesn't have to parse a format string.
msg282783 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-12-09 13:45
Minimum patch for Python 3.6: only modify calls using the "O" format.
msg283250 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-12-15 08:09
New changeset f2745b64d8b7 by Victor Stinner in branch '3.6':
_asyncio uses _PyObject_CallMethodIdObjArgs()
https://hg.python.org/cpython/rev/f2745b64d8b7
History
Date User Action Args
2016-12-15 08:10:13vstinnersetstatus: open -> closed
resolution: fixed
2016-12-15 08:09:39python-devsetnosy: + python-dev
messages: + msg283250
2016-12-09 13:45:42vstinnersetfiles: + _asyncio.patch
keywords: + patch
messages: + msg282783
2016-12-09 13:44:17vstinnersetmessages: + msg282781
2016-12-09 13:38:38vstinnersetmessages: + msg282779
2016-12-09 13:38:13vstinnercreate