Title: Enabling access to showsyntaxerror for IDLE's shell
Type: enhancement Stage:
Components: IDLE Versions: Python 3.11, Python 3.10
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: aroberge, terry.reedy
Priority: normal Keywords:

Created on 2021-03-11 20:38 by aroberge, last changed 2021-06-28 21:38 by aroberge.

Messages (9)
msg388524 - (view) Author: Andre Roberge (aroberge) * Date: 2021-03-11 20:38
As a result of, IDLE now supports custom exception hook for almost all cases except for code entered in the interactive shell that result in SyntaxError. It would be useful for some applications if a way to replace the error handling currently done by IDLE for this particular case was made available to user code.

I consider this to be in the "would be nice to have" category, but in no means a high priority item.
msg396152 - (view) Author: Andre Roberge (aroberge) * Date: 2021-06-19 19:57
Would it be possible to add a single line of code to idlelib's, as indicated below:

def showsyntaxerror(self, filename=None):
    """... """
    linecache.cache["<SyntaxError>"] = linecache.cache[filename]  # here
    tkconsole = self.tkconsole

Of course, another name than "<SyntaxError>" could be chosen.

This would allow users (like me, with Friendly) to retrieve the code that caused a SyntaxError and process it as they wish. In my case, it would allow me to reproduce the SyntaxError and have Friendly provide an explanation and possible suggestions for fixing the error.

This would complement the new support for user defined sys.excepthook introduced in Python 3.10.
msg396160 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-06-20 02:21
The comment should be "# For 3rd party use.  See issue 43476."
Updates to the pypi url, now should be recorded here on this issue. 

The undocumented name 'cache' is excluded from linecache.__all__ and therefore private.  We use it at our own risk (which should be extremely minimal).

I cannot think of anyway that adding another '<...>' pseudoname entry could hurt.  The only possibility is adding another copy of an entry, but they are small.

But are you sure that this is value for the key?  We could instead make the pseudotext be anything we want, such as a serialized version of the exception instance itself, including the error line.  Why reproduce the error instead of fetching it?  (If you do reproduce, you have to make sure you use the same compile options.)
msg396166 - (view) Author: Andre Roberge (aroberge) * Date: 2021-06-20 09:36
I just thought of making a copy of the file content under a known name as the least disruptive approach for IDLE's code - even less so than the recent addition required to support user-defined exception hooks.

For SyntaxErrors and subclasses [1], Friendly needs the source code as well as the error message, offset, lineno (end_offset and end_lineno if they are present) in order to perform its analysis. In all cases (i.e not only with IDLE but with other environments like Jupyter notebooks), it retrieves this source code from the given filename using the linecache module.  

If it were possible to get those exception arguments using RPC, this would be much better than what I suggested. This might also be much more useful that what I suggest for any third-party code wanting access to the traceback information for SyntaxErrors. However, I could not find a simple way of doing this.

[1] The required information for run-time errors, such as NameError, etc., is already available with sys.excepthook.
msg396204 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-06-21 03:45
What I am proposing that pseudofile <SyntaxError> have one line representing a tuple with all the exception information, *including the filename* for the code with the error.  In Shell, the filename will usually be another pseudofile name, <pyshell#xx>. 

The latter are set with
    def stuffsource(self, source):
        "Stuff source in the filename cache"
        filename = "<pyshell#%d>" % self.gid
        self.gid = self.gid + 1
        lines = source.split("\n")
        linecache.cache[filename] = len(source)+1, 0, lines, filename
        return filename

I think the +1 is for a '\n' that will be appended.  The linecache line is otherwise our model.

The following is how I created a line <SyntaxError> while testing.

try: compile('a b', '<pyshell#33>', 'single')
except SyntaxError as e:
     err = str((type(e).__name__, e.args[0], *e.args[1]))+'\n'

err will be the single line for the file:
"('SyntaxError', 'invalid syntax. Perhaps you forgot a comma?', '<pyshell#33>', 1, 1, 'a b', 1, 4)"

For the patch, err would can be calculated a little differently further down in showsyntaxerror (which needs updating).

Then set the cache with 
  linecache.cache["<SyntaxError>"] = (len(err), 0, [err], "<SyntaxError>")

In friendly, retrieve the lines and unpack the evaluated tuple (\n at the end is ok).

exception, message, filename, line, col, text, line_end, col_end = eval(lines[0])

Use filename to retrieve the error code lines as you wish.
msg396214 - (view) Author: Andre Roberge (aroberge) * Date: 2021-06-21 09:06
Having a fake file with a single line containing all the exception information as described would definitely provide all the required information needed. It would be much better than what I had suggested previously.
msg396215 - (view) Author: Andre Roberge (aroberge) * Date: 2021-06-21 09:24
Unless I am mistaken, when compiling the code, both SyntaxError and OverflowError instances are caught at the same stage, and likely passed on to showsyntaxerror.  For OverflowError, e.args is not normally a tuple but a string, and *e.args[1] would raise an exception.
msg396354 - (view) Author: Andre Roberge (aroberge) * Date: 2021-06-22 17:31
I can confirm that, if the existing line

type, value, tb = sys.exc_info()

in pyshell.ModifiedInterpreter.showsyntaxerror is replaced by

e_type, value, tb = sys.exc_info()
err = str((type(e_type).__name__, *value)+'\n'
linecache.cache["<SyntaxError>"] = (len(err), 0, [err], "<SyntaxError>")

then all the required information to analyze the cause of the error can be retrieved using rpc.

Aside: In doing various tests, I noticed that "IndentationError" are simply reported as "SyntaxError" in Idle, since at least Python 3.6.
This is just a cosmetic issue.
msg396664 - (view) Author: Andre Roberge (aroberge) * Date: 2021-06-28 21:38
I made an error in my previous comment: the code I wrote is obviously incorrect as it contains a SyntaxError.
Date User Action Args
2021-06-28 21:38:36arobergesetmessages: + msg396664
2021-06-22 17:31:42arobergesetmessages: + msg396354
2021-06-21 09:24:46arobergesetmessages: + msg396215
2021-06-21 09:06:23arobergesetmessages: + msg396214
2021-06-21 03:45:57terry.reedysetmessages: + msg396204
2021-06-20 09:36:04arobergesetmessages: + msg396166
2021-06-20 02:21:31terry.reedysetmessages: + msg396160
versions: + Python 3.11
2021-06-19 19:57:21arobergesetmessages: + msg396152
2021-03-11 20:38:57arobergecreate