This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Possible concurrency bug in asyncio, AttributeError in tasks.py
Type: Stage: resolved
Components: Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Jack.Murray, giampaolo.rodola, gvanrossum, pitrou, python-dev, vstinner, yselivanov
Priority: normal Keywords:

Created on 2014-04-24 02:13 by Jack.Murray, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (6)
msg217112 - (view) Author: Jack Murray (Jack.Murray) Date: 2014-04-24 02:13
AttributeError in /usr/lib/python3.4/asyncio/tasks.py feels like it might be a concurrency issue. I can't reproduce it, and my python isn't good enough to know how to simulate raising the exception at a random time during the execution of the program. Here's the PoC:

import asyncio
import sys
from asyncio import async
import time
import random
asyncio.tasks._DEBUG = True


loop = asyncio.get_event_loop()

def read_something():
  print(input())

@asyncio.coroutine
def func(arg):
  while True:
    sys.stdout.write("\rtest"+str(arg))
    yield from asyncio.sleep(0)

loop.add_reader(sys.stdin, read_something)
loop.call_soon(async, func(1))
loop.call_soon(async, func(2))
loop.call_soon(async, func(3))
loop.call_soon(async, func(4))

time.sleep(1)

try:
  loop.run_forever()
except KeyboardInterrupt:
  print("handled\n")
  pass
finally:
  pass

and here is the stack trace:

ktn:~/ $ python asynctest.py                                                                                       [11:55:03]
test3^Chandled

Future/Task exception was never retrieved
future: Task(<func>)<exception=KeyboardInterrupt()>
Traceback (most recent call last):
  File "asynctest.py", line 29, in <module>
    loop.run_forever()
  File "/usr/lib/python3.4/asyncio/base_events.py", line 184, in run_forever
    self._run_once()
  File "/usr/lib/python3.4/asyncio/base_events.py", line 800, in _run_once
    handle._run()
  File "/usr/lib/python3.4/asyncio/events.py", line 39, in _run
    self._callback(*self._args)
  File "/usr/lib/python3.4/asyncio/tasks.py", line 337, in _wakeup
    self._step(value, None)
  File "/usr/lib/python3.4/asyncio/tasks.py", line 283, in _step
    result = next(coro)
  File "/usr/lib/python3.4/asyncio/tasks.py", line 50, in __next__
    return next(self.gen)
  File "asynctest.py", line 18, in func
    yield from asyncio.sleep(0)
  File "/usr/lib/python3.4/asyncio/tasks.py", line 94, in wrapper
    w = CoroWrapper(coro(*args, **kwds), func)
  File "/usr/lib/python3.4/asyncio/tasks.py", line 42, in __init__
    assert inspect.isgenerator(gen), gen
KeyboardInterrupt
Exception ignored in: <bound method CoroWrapper.__del__ of <asyncio.tasks.CoroWrapper object at 0x7f0dedaf79d8>>
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/tasks.py", line 62, in __del__
    frame = self.gen.gi_frame
AttributeError: gen
msg217133 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2014-04-24 17:32
Looks like there is a bug in CoroWrapper -- when the assert in __init__ fails, __del__ gets called imediately after and that triggers this traceback.

However I'm not sure what causes the assert to fail -- it looks like this is coming from sleep(), which does not make sense to me.
msg217155 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2014-04-25 06:45
Oh wait, it looks like the assert failed because KeyboardInterrupt hit right at that point. I ran the program a few times and when I hit ^C I get a traceback at a different point in the code each time. This is as expected. You must have hit the rare case where it hit right at the assert -- then the behavior I described can happen.

Anyway, I think this would fix it:

--- a/asyncio/tasks.py  Fri Apr 18 09:51:35 2014 -0700
+++ b/asyncio/tasks.py  Thu Apr 24 23:44:57 2014 -0700
@@ -76,7 +76,8 @@
         return self.gen.gi_code

     def __del__(self):
-        frame = self.gen.gi_frame
+        gen = getattr(self, 'gen', None)
+        frame = getattr(gen, 'gi_frame', None)
         if frame is not None and frame.f_lasti == -1:
             func = self.func
             code = func.__code__
msg217296 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-04-27 17:47
New changeset d42d3d3f9c41 by Guido van Rossum in branch '3.4':
asyncio: Be careful accessing instance variables in __del__ (closes #21340).
http://hg.python.org/cpython/rev/d42d3d3f9c41

New changeset 0cb436c6f082 by Guido van Rossum in branch 'default':
Merge 3.4 -> default: asyncio: Be careful accessing instance variables in __del__ (closes #21340).
http://hg.python.org/cpython/rev/0cb436c6f082
msg217327 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-04-27 23:22
Why not using try/except AttributeError?
msg217340 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2014-04-28 04:03
Sorry, should have let someone review it. I'm a bit out of practice. :-)

But in this case I think three-arg getattr() is better; less code, less indentation, and the final question (is the frame not None?) must still be asked.
History
Date User Action Args
2022-04-11 14:58:02adminsetgithub: 65539
2014-04-28 04:03:26gvanrossumsetmessages: + msg217340
2014-04-27 23:22:50vstinnersetmessages: + msg217327
2014-04-27 17:47:36python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg217296

resolution: fixed
stage: resolved
2014-04-25 06:45:16gvanrossumsetmessages: + msg217155
2014-04-24 17:32:43gvanrossumsetmessages: + msg217133
2014-04-24 17:15:23berker.peksagsetnosy: + gvanrossum, pitrou, vstinner, giampaolo.rodola, yselivanov
2014-04-24 02:15:14Jack.Murraysettitle: Possible bug in asyncio -> Possible concurrency bug in asyncio, AttributeError in tasks.py
2014-04-24 02:13:42Jack.Murraycreate