The PEP 657 introduced ^^^ in tracebacks. It is useful when the error happens on an sub-expression in a long line. Example:

  File "/home/vstinner/python/main/Lib/ftplib.py", line 462, in retrlines
    with self.transfercmd(cmd) as conn, \
         ^^^^^^^^^^^^^^^^^^^^^

But ^^^ makes the output more verbose and doesn't bring much value when the error concerns the whole line:

  File "/home/vstinner/python/main/Lib/socket.py", line 845, in create_connection
    raise err
    ^^^^^^^^^

Would it be possible to omit ^^^ when it concerns the whole line?

Full example (currently):

ERROR: test_retrlines (test.test_ftplib.TestFTPClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/test_ftplib.py", line 603, in test_retrlines
    self.client.retrlines('retr', received.append)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ftplib.py", line 462, in retrlines
    with self.transfercmd(cmd) as conn, \
         ^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ftplib.py", line 393, in transfercmd
    return self.ntransfercmd(cmd, rest)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ftplib.py", line 354, in ntransfercmd
    conn = socket.create_connection((host, port), self.timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/socket.py", line 845, in create_connection
    raise err
    ^^^^^^^^^
  File "/home/vstinner/python/main/Lib/socket.py", line 833, in create_connection
    sock.connect(sa)
    ^^^^^^^^^^^^^^^^
ConnectionRefusedError: [Errno 111] Connection refused

I would prefer:

ERROR: test_retrlines (test.test_ftplib.TestFTPClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/test_ftplib.py", line 603, in test_retrlines
    self.client.retrlines('retr', received.append)
  File "/home/vstinner/python/main/Lib/ftplib.py", line 462, in retrlines
    with self.transfercmd(cmd) as conn, \
         ^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ftplib.py", line 393, in transfercmd
    return self.ntransfercmd(cmd, rest)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ftplib.py", line 354, in ntransfercmd
    conn = socket.create_connection((host, port), self.timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/socket.py", line 845, in create_connection
    raise err
  File "/home/vstinner/python/main/Lib/socket.py", line 833, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused


In term of release process, can we change the traceback after Python 3.10.0 final? Or can we only change it in Python 3.11? I mark the issue as a release blocker, but I let Pablo (author of the PEP and Python 3.10 release manager) decide.

I looked at the implementation. It's more complex than what I expected.

tb_displayline(): _Py_DisplaySourceLine() returns the source line with the indentation, but then it truncates the indentation. 

extract_anchors_from_line() includes the indentation if I understand correctly.

There is also a special case for trailing spaces. Like this code path:

        // If this is a multi-line expression, then we will highlight until
        // the last non-whitespace character.

There is also a funny part about bytes vs Unicode vs UTF-8, that I don't get.

> In term of release process, can we change the traceback after Python 3.10.0 final? Or can we only change it in Python 3.11?

I don't follow what you want to change in 3.10.0 final, PEP 657 is for Python 3.11. Can you clarify?

Regarding the issue: I understand what you mean but I don't think we should get into suppressing the indicators in special situations. Some people may say that in a line that assigns to a call:

x = foo(x)
    ^^^^^^

highlighting foo() is also not very valuable but this case is much harder to detect so I don't want to open the window to "special cases".

Also, it will be very confusing to users why some lines are highlighted and why some lines aren't.

So, in general, I feel we should not do this, but I am open to more discussion about it.

> I don't follow what you want to change in 3.10.0 final, PEP 657 is for Python 3.11. Can you clarify?

Ah, I thaught that the PEP was implemented in 3.10, I didn't notice that it's a new feature of Python 3.11 :-)

> Regarding the issue: I understand what you mean but I don't think we should get into suppressing the indicators in special situations.

In my example, there are 6 frames. The indicators cover the whole line of 3 frames: 50%. It's a common case, it's not special.

Python tracebacks are already very verbose: a single exception can easily take 20-30 lines. It's worse for chained exceptions. Python 3.11 makes it worse: up to 1.5x longer.

>In my example, there are 6 frames. The indicators cover the whole line of 3 frames: 50%. It's a common case, it's not special.

Special is not uncommon, is just some different behaviour rather than the default one. I am not arguing that is a rare scenario, I am arguing that is a special case.

> Python 3.11 makes it worse: up to 1.5x longer

Sound like you likely want to use the environment variable to deactivate the extra information ;)

It seems that I will have to learn to use PYTHONNODEBUGRANGES=1 and -Xno_debug_ranges. I close the issue.