classification
Title: Frame with -1 line number
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Mark.Shannon Nosy List: Anthony Sottile, Mark.Shannon, domdfcoding, gvanrossum, pablogsal, shreyanavigyan, vstinner
Priority: normal Keywords: patch

Created on 2021-06-03 01:51 by vstinner, last changed 2021-08-13 08:27 by Mark.Shannon. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26801 merged Mark.Shannon, 2021-06-19 11:45
PR 26821 merged Mark.Shannon, 2021-06-21 10:02
PR 26890 merged Mark.Shannon, 2021-06-24 09:30
PR 26891 merged Mark.Shannon, 2021-06-24 09:37
Messages (16)
msg394969 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-03 01:51
While debugging https://bugs.python.org/issue43921 on Windows, I got a traceback with a single frame and the frame line number is -1.

It looks like a Python 3.11 regression.

Mark, Guido: can it be related to recent optimization work done in the main branch?

See also bpo-44288 "unittest: _is_relevant_tb_level() fails because tb.tb_frame.f_globals=None".

======================================================================
FAIL: test_pha_required_nocert (test.test_ssl.TestPostHandshakeAuth)
----------------------------------------------------------------------
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2522)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\vstinner\python\main\lib\test\test_ssl.py", line -1, in test_pha_required_nocert
AssertionError: "certificate required" does not match "EOF occurred in violation of protocol (_ssl.c:2522)"
msg395209 - (view) Author: Dominic Davis-Foster (domdfcoding) * Date: 2021-06-06 18:12
Is this a re-regression of https://bugs.python.org/issue43933?
msg396110 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2021-06-19 00:15
this appears to have regressed in 3.10 as well according to some reports on pytest: https://github.com/pytest-dev/pytest/pull/8227#issuecomment-864327090
msg396111 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2021-06-19 00:30
here's the traceback pytest is trying to display and crashing:

```
Traceback (most recent call last):
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/runner.py", line 311, in from_call
    result: Optional[TResult] = func()
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/runner.py", line 341, in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", line 503, in collect
    self._inject_setup_module_fixture()
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", line 516, in _inject_setup_module_fixture
    self.obj, ("setUpModule", "setup_module")
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", line 291, in obj
    self._obj = obj = self._getobj()
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", line 500, in _getobj
    return self._importtestmodule()
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", line 578, in _importtestmodule
    mod = import_path(self.fspath, mode=importmode)
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/pathlib.py", line 524, in import_path
    importlib.import_module(module_name)
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/tmp/rinohtype/tests/test_attribute.py", line 11, in <module>
    from rinoh.attribute import Attribute, Bool
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/__init__.py", line 41, in <module>
    from . import resource
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py", line 205, in <module>
    from .template import DocumentTemplate
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/template.py", line 42, in <module>
    from .stylesheets import sphinx
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/stylesheets/__init__.py", line 42, in <module>
    .format(stylesheet.description, stylesheet))
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/style.py", line 670, in __str__
    for name, entry_point in self.installed_resources:
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py", line 54, in installed_resources
    for entry_point in ilm.entry_points()[cls.entry_point_group]:
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line 979, in entry_points
    return SelectableGroups.load(eps).select(**params)
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line 437, in load
    ordered = sorted(eps, key=by_group)
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line -1, in <genexpr>
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen
    k = key(element)
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line 600, in _normalized_name
    return Prepared.normalize(self.name)
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line 841, in normalize
    return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/re.py", line 187, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
```
msg396118 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2021-06-19 06:30
here is a minimal reproduction:

```python
def iterboom():
    raise AssertionError
    yield 1

next(1 for x in iterboom())
```

python 3.9:


```
$ python3.9 t.py
Traceback (most recent call last):
  File "/tmp/rinohtype/t.py", line 5, in <module>
    next(1 for x in iterboom())
  File "/tmp/rinohtype/t.py", line 5, in <genexpr>
    next(1 for x in iterboom())
  File "/tmp/rinohtype/t.py", line 2, in iterboom
    raise AssertionError
AssertionError
```

```
$ python3.10 t.py
Traceback (most recent call last):
  File "/tmp/rinohtype/t.py", line 5, in <module>
    next(1 for x in iterboom())
  File "/tmp/rinohtype/t.py", line -1, in <genexpr>
  File "/tmp/rinohtype/t.py", line 2, in iterboom
    raise AssertionError
AssertionError
```
msg396128 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-19 11:25
Thanks Anthony, that's a big help.
msg396216 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-21 09:55
New changeset 82e5c28af7049c4f5343c808f172cbe2e145f49b by Mark Shannon in branch 'main':
bpo-44297: Fix missing line number in generator expressions (GH-26801)
https://github.com/python/cpython/commit/82e5c28af7049c4f5343c808f172cbe2e145f49b
msg396224 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-21 10:47
New changeset 7674c83d81905d6afe989ca3f93f08b7939b057c by Mark Shannon in branch '3.10':
bpo-44297: Fix missing line number in generator expressions (GH-26821)
https://github.com/python/cpython/commit/7674c83d81905d6afe989ca3f93f08b7939b057c
msg396255 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-21 15:42
Can the issue be closed?

I'm not sure which assertion failed in test_ssl.test_pha_required_nocert(). I bet on this one:

                with self.assertRaisesRegex(
                    ssl.SSLError,
                    '(certificate required|EOF occurred)'
                ):

I don't know if the commit 82e5c28af7049c4f5343c808f172cbe2e145f49b is supported is to fix issue or not.
msg396257 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-21 15:44
Oh, I can still reproduce the issue on the main branch with this patch:

diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
index cee97a8302..3f66818ae1 100644
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -4499,7 +4499,7 @@ def msg_cb(conn, direction, version, content_type, msg_type, data):
                 # server aborts connection with an error.
                 with self.assertRaisesRegex(
                     ssl.SSLError,
-                    '(certificate required|EOF occurred)'
+                    'xxxxxxxxxxxxx'
                 ):
                     # receive CertificateRequest
                     data = s.recv(1024)



$ ./python -m test test_ssl -m test_pha_required_nocert -v
(...)
FAIL: test_pha_required_nocert (test.test_ssl.TestPostHandshakeAuth)
----------------------------------------------------------------------
(...)
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/test_ssl.py", line -1, in test_pha_required_nocert
AssertionError: (...)
...


=> "line -1"
msg396361 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-06-22 20:09
Here presumably the error is occurring somewhere near unittest. I've tested this and wrote a minimal reproducible example. 

```
import unittest

class TestingError(unittest.TestCase):
    def test_negative_one(self):
        with self.assertRaisesRegex(AssertionError, "xxxxx"):
            self.assertEqual(1, 2)
```

Running this with `unittest discover` or in any other command-line way (like -c) results in frame with -1 while running it from script (with unittest.main probably) does not.

Hope this helps.
msg396389 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-06-23 08:59
Oh it is also occurring when running from script (I ran it from IDLE and it always results in correct lineno). I wrote example similar to how unittests work and the frame with -1 lineno is also occurring there. Here is the code -

```
class A:
    def __enter__(self):
        return self
    def __exit__(self, *args, **kwargs):
        raise Exception("Frame is -1 if program is run from command line")

with A():
    raise Exception("Normal Error")
```

Also, unsurprisingly, pdb fails with a "TypeError: '>=' not supported between instances of 'NoneType' and 'int'". Full pdb log -

Traceback (most recent call last):
  File "C:\Users\shrey\Desktop\line_negative_one.py", line -1, in <module>
  File "C:\github\cpython\lib\bdb.py", line 96, in trace_dispatch
    return self.dispatch_exception(frame, arg)
  File "C:\github\cpython\lib\bdb.py", line 169, in dispatch_exception
    if self.stop_here(frame):
  File "C:\github\cpython\lib\bdb.py", line 212, in stop_here
    return frame.f_lineno >= self.stoplineno
TypeError: '>=' not supported between instances of 'NoneType' and 'int'
msg396430 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-23 15:57
Thanks for the reproducer.
msg396456 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-06-24 07:03
Found the culprit - https://github.com/python/cpython/blob/769d7d0c66c5b86e2dd29b9ce67ac2daaab1bb38/Python/compile.c#L5268

Same goes for async with - https://github.com/python/cpython/blob/769d7d0c66c5b86e2dd29b9ce67ac2daaab1bb38/Python/compile.c#L5171
(Didn't test async with so I can be wrong there.)

Is this intentional? I removed these lines and the line number is coming correct. Can changing this affect other use cases?

Without the change:


Traceback (most recent call last):
  File "C:\Users\shrey\Desktop\line_negative_one.py", line 8, in <module>
    raise Exception("Normal Error")
Exception: Normal Error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\shrey\Desktop\line_negative_one.py", line -1, in <module>
  File "C:\Users\shrey\Desktop\line_negative_one.py", line 5, in __exit__
    raise Exception("Frame is -1 if program is run from command line")
Exception: Frame is -1 if program is run from command line



With the change:



Traceback (most recent call last):
  File "C:\Users\shrey\Desktop\line_negative_one.py", line 8, in <module>
    raise Exception("Normal Error")
Exception: Normal Error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\shrey\Desktop\line_negative_one.py", line 7, in <module>
    with A():
  File "C:\Users\shrey\Desktop\line_negative_one.py", line 5, in __exit__
    raise Exception("Frame is -1 if program is run from command line")
Exception: Frame is not -1 anymore if program is run from command line
msg396475 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-24 12:09
New changeset 0b6b2865187bca7ed7f1f511a02fc8bd13ee38ca by Mark Shannon in branch '3.10':
bpo-44297: Add a regression test for line numbers in with statements (GH-26891)
https://github.com/python/cpython/commit/0b6b2865187bca7ed7f1f511a02fc8bd13ee38ca
msg399501 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-08-13 08:27
I believe this to be fixed.
History
Date User Action Args
2021-08-13 08:27:24Mark.Shannonsetstatus: open -> closed
resolution: fixed
messages: + msg399501

stage: patch review -> resolved
2021-06-24 12:09:21pablogsalsetmessages: + msg396475
2021-06-24 09:37:18Mark.Shannonsetpull_requests: + pull_request25468
2021-06-24 09:30:59Mark.Shannonsetpull_requests: + pull_request25467
2021-06-24 07:03:44shreyanavigyansetmessages: + msg396456
2021-06-23 15:57:52Mark.Shannonsetassignee: Mark.Shannon
messages: + msg396430
2021-06-23 08:59:12shreyanavigyansetmessages: + msg396389
2021-06-22 20:09:50shreyanavigyansetnosy: + shreyanavigyan
messages: + msg396361
2021-06-21 15:44:33vstinnersetmessages: + msg396257
2021-06-21 15:42:17vstinnersetmessages: + msg396255
2021-06-21 10:47:44Mark.Shannonsetmessages: + msg396224
2021-06-21 10:02:09Mark.Shannonsetpull_requests: + pull_request25402
2021-06-21 09:55:30Mark.Shannonsetmessages: + msg396216
2021-06-19 11:45:20Mark.Shannonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25382
2021-06-19 11:25:35Mark.Shannonsetmessages: + msg396128
2021-06-19 06:30:39Anthony Sottilesetmessages: + msg396118
2021-06-19 00:30:43Anthony Sottilesetmessages: + msg396111
2021-06-19 00:15:38Anthony Sottilesetnosy: + Anthony Sottile, pablogsal

messages: + msg396110
versions: + Python 3.10
2021-06-06 18:12:37domdfcodingsetnosy: + domdfcoding
messages: + msg395209
2021-06-03 01:51:01vstinnercreate