classification
Title: linecache.getline TypeError when formatting tracebacks in stacks containing an async list comprehension
Type: Stage: resolved
Components: Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: FFY00, Mark.Shannon, graingert, iritkatriel, nedbat, pablogsal
Priority: Keywords: patch

Created on 2021-06-17 18:49 by graingert, last changed 2021-07-08 16:48 by pablogsal. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26781 merged FFY00, 2021-06-17 21:41
PR 26782 closed FFY00, 2021-06-18 00:02
PR 27072 merged pablogsal, 2021-07-08 16:31
Messages (11)
msg396014 - (view) Author: Thomas Grainger (graingert) * Date: 2021-06-17 18:49
demo:

import traceback
import io

async def foo():
    yield 1
    traceback.print_stack(file=io.StringIO())
    yield 2

async def bar():
    return [chunk async for chunk in foo()]


next(bar().__await__(), None)
print("working!")


Traceback (most recent call last):
  File "/home/graingert/projects/anyio/foo.py", line 13, in <module>
    next(bar().__await__(), None)
  File "/home/graingert/projects/anyio/foo.py", line 10, in bar
    return [chunk async for chunk in foo()]
  File "/home/graingert/projects/anyio/foo.py", line -1, in <listcomp>
  File "/home/graingert/projects/anyio/foo.py", line 6, in foo
    traceback.print_stack(file=io.StringIO())
  File "/usr/lib/python3.10/traceback.py", line 203, in print_stack
    print_list(extract_stack(f, limit=limit), file=file)
  File "/usr/lib/python3.10/traceback.py", line 224, in extract_stack
    stack = StackSummary.extract(walk_stack(f), limit=limit)
  File "/usr/lib/python3.10/traceback.py", line 379, in extract
    f.line
  File "/usr/lib/python3.10/traceback.py", line 301, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()
  File "/usr/lib/python3.10/linecache.py", line 31, in getline
    if 1 <= lineno <= len(lines):
TypeError: '<=' not supported between instances of 'int' and 'NoneType'
msg396022 - (view) Author: Filipe Laíns (FFY00) * (Python triager) Date: 2021-06-17 21:43
I bissected this to 088a15c49d99ecb4c3bef93f8f40dd513c6cae3b and submitted a patch making traceback.FrameSummary take into consideration that lineno might be None, I believe this is probably the correct fix.
msg396026 - (view) Author: Filipe Laíns (FFY00) * (Python triager) Date: 2021-06-18 00:03
Upon further investigation, there are actually two issues here. The first would be the one I identified already, traceback.FrameSummary not being prepared for lineno being None, but there is also a regression in lineno being invalid in this situation in the first place.

With only GH-26781, the traceback will look like the following:


  File "/home/anubis/git/cpython/rep.py", line 13, in <module>
    next(bar().__await__(), None)
  File "/home/anubis/git/cpython/rep.py", line 10, in bar
    return [chunk async for chunk in foo()]
  File "/home/anubis/git/cpython/rep.py", line None, in <listcomp>
  File "/home/anubis/git/cpython/rep.py", line 6, in foo
    traceback.print_stack()
working!


which is different from 3.9



  File "/home/anubis/git/cpython/rep.py", line 13, in <module>
    next(bar().__await__(), None)
  File "/home/anubis/git/cpython/rep.py", line 10, in bar
    return [chunk async for chunk in foo()]
  File "/home/anubis/git/cpython/rep.py", line 10, in <listcomp>
    return [chunk async for chunk in foo()]
  File "/home/anubis/git/cpython/rep.py", line 6, in foo
    traceback.print_stack()
working!


I bisected the second issue to b37181e69209746adc2119c471599a1ea5faa6c8 which moves generators to bytecode, and when doing so changes the behavior to set lineno to -1. I have opened a GH-26782 to fixing this.
msg396101 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-18 21:59
Mark, can you take a look at this?
msg396230 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-21 11:19
This appears to be a duplicate of https://bugs.python.org/issue44297
msg396231 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-21 11:21
With the latest 3.10, I get:

  File "/home/mark/test/test.py", line 13, in <module>
    next(bar().__await__(), None)
  File "/home/mark/test/test.py", line 10, in bar
    return [chunk async for chunk in foo()]
  File "/home/mark/test/test.py", line 10, in <listcomp>
    return [chunk async for chunk in foo()]
  File "/home/mark/test/test.py", line 6, in foo
    traceback.print_stack()
working!


Thomas, can you confirm?
msg397135 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-08 10:56
Beta 4 is in a few days. Can someone confirm of this is fixed or if it still needs a patch?
msg397140 - (view) Author: Filipe Laíns (FFY00) * (Python triager) Date: 2021-07-08 13:24
The issue that made the line number be missing is fixed, but GH-26781 is needed to account for 088a15c49d99ecb4c3bef93f8f40dd513c6cae3b, even though it is no longer triggered in this situation.
msg397149 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-08 16:28
New changeset 91a8f8c16ca9a7e2466a8241d9b41769ef97d094 by Filipe Laíns in branch 'main':
bpo-44446: support lineno being None in traceback.FrameSummary (GH-26781)
https://github.com/python/cpython/commit/91a8f8c16ca9a7e2466a8241d9b41769ef97d094
msg397155 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-08 16:47
New changeset 61eb9b5dfd919ba5d1ec9f7df0137f2e6d196972 by Pablo Galindo in branch '3.10':
[3.10] bpo-44446: support lineno being None in traceback.FrameSummary (GH-26781) (GH-27072)
https://github.com/python/cpython/commit/61eb9b5dfd919ba5d1ec9f7df0137f2e6d196972
msg397156 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-08 16:48
Feel free to reopen if we are missing anything
History
Date User Action Args
2021-07-08 16:48:18pablogsalsetmessages: + msg397156
2021-07-08 16:48:09pablogsalsetpriority: release blocker ->
2021-07-08 16:47:55pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-07-08 16:47:21pablogsalsetmessages: + msg397155
2021-07-08 16:31:34pablogsalsetpull_requests: + pull_request25623
2021-07-08 16:28:10pablogsalsetmessages: + msg397149
2021-07-08 13:24:32FFY00setmessages: + msg397140
2021-07-08 10:56:53pablogsalsetmessages: + msg397135
2021-06-21 11:21:27Mark.Shannonsetmessages: + msg396231
2021-06-21 11:19:03Mark.Shannonsetmessages: + msg396230
2021-06-18 22:00:31pablogsalsetpriority: normal -> release blocker
2021-06-18 21:59:35pablogsalsetnosy: + pablogsal

messages: + msg396101
versions: + Python 3.11, - Python 3.10
2021-06-18 00:03:13FFY00setmessages: + msg396026
2021-06-18 00:02:59FFY00setpull_requests: + pull_request25367
2021-06-17 21:43:42FFY00setmessages: + msg396022
2021-06-17 21:41:31FFY00setkeywords: + patch
nosy: + FFY00

pull_requests: + pull_request25366
stage: patch review
2021-06-17 20:45:38FFY00setnosy: + iritkatriel
2021-06-17 19:08:23graingertsetnosy: + nedbat
2021-06-17 19:07:58graingertsetnosy: + Mark.Shannon, - nedbat
2021-06-17 19:04:38nedbatsetnosy: + nedbat
2021-06-17 18:49:12graingertcreate