classification
Title: Custom error messages when print & exec are used as statements
Type: enhancement Stage: resolved
Components: Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alex, glyph, gvanrossum, ncoghlan, python-dev
Priority: normal Keywords: patch

Created on 2014-06-05 15:07 by ncoghlan, last changed 2014-08-22 11:00 by ncoghlan. This issue is now closed.

Files
File name Uploaded Description Edit
custom_builtin_syntax_errors.diff ncoghlan, 2014-06-05 15:07 Custom syntax errors for print and exec used as statements review
issue21669_custom_error_messages.diff ncoghlan, 2014-06-06 12:57 Simpler approach based on a heuristic in SyntaxError.__init__ review
issue21669_custom_error_messages_v2.diff ncoghlan, 2014-06-07 12:37 Also handle the compound statement cases review
Messages (13)
msg219816 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-06-05 15:07
I realised my experiment with supporting implicit calls could potentially be used as the basis for a patch that reported more specific error details when "print" and "exec" were used as statements, so I went ahead and updated it to do so.

The initial patch has a failure in test_source_encoding - this appears to be a general issue with moving syntax errors from the parser (which reliably sets the "text" attribute on the raised SyntaxError) to the AST compiler (which sets that to None when running from a string compiled directly from memory rather than from a file on disk).

I've also only flagged this as a patch for 3.5 - I can't think of a way to do it without changing the language grammar to allow the error to be generated in a later stage of the compilation process, and that's not the kind of thing we want to be doing in a maintenance release.
msg219822 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2014-06-05 16:15
I'm sorry, but I find this way too intrusive, and a little risky too (I'm not sure how to verify even that the new parser accepts exactly the same set of programs as the old version).

I would much prefer a solution to this particular issue along the lines of a heuristic based on detecting whether the line where the syntax error occurs starts with the token 'print' or 'exec' followed by anything except a left parenthesis.

Such a heuristic could also be added to 3.4.
msg219854 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-06-05 23:08
As in, putting something either in the SyntaxError constructor or else in
the parser code that emits them? I like that - the fact my initial approach
broke a test was rather concerning, and a change purely on the error
handling side should be much safer.
msg219855 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2014-06-05 23:14
Yes, something like that. Don't change the grammar, just hack the heck out of the error message.
msg219880 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-06-06 12:57
Heuristic based approach that just does a fairly simple check for the syntax error text starting with "print " or "exec " when the text doesn't contain a left parenthesis.

This will still miss a few cases where the left parenthesis is inside a larger expression (like a string, list or dict), but that extra check avoids false triggering on cases like "print (a.)".
msg219890 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2014-06-06 16:29
Nice! I put it through a bit of a torture test and found a few odd corners. E.g. it doesn't catch this:

    if 1: print 42

nor this:

    if 1:
        print 42

nor this:

    def foo():
        print 42

I also notice that if the printed expression starts with a unary + or -, it is not a syntax error but a type error. But I don't think we should try to do anything about that.
msg219892 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2014-06-06 16:46
I also found some amusing false positives (syntax errors that weren't valid print statements in Python 2):

    print [/
    print /
    print )    # but not "print)" !
    print]

None of these matter though.  Perhaps more concerning is how many things are valid syntax, despite making little sense:

    print [42]
    print

Still I like the idea -- even if it only catches 50% of all print statements that would still be a huge win. (And I think it's probably closer to 80%.)
msg219932 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-06-07 12:37
Updated patch with the heuristics factored out into a helper function, with a more detailed explanation and additional logic to handle compound statements.

>>> def foo():
...     print bar
  File "<stdin>", line 2
    print bar
            ^
SyntaxError: Missing parentheses in call to 'print'

It's still just basic string hackery, though. The one liner handling, for example, relies on the fact that ":<whitespace>print " and ":<whitespace>exec " are going to be uncommon outside Python 2 code being ported to Python 3, so it just looks for the first colon on the line and checks from there, without worrying about slice notation or dicts.
msg220048 - (view) Author: Glyph Lefkowitz (glyph) Date: 2014-06-08 19:32
Just my 2ยข here: rather than debating cases in the abstract, it would be interesting to 'pip install' a couple of popular 2.x-only packages and see if the error message is an improvement.

My experience is that learners don't hit this so much by writing their own code wrong, but by loading a dependency with incorrect metadata on the wrong Python.  (Which suggests to me that a URL in the error message telling you how to download a different version of Python would be very helpful as well.)
msg220077 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-06-09 01:32
My main aim here is to offer a custom error message that can be looked up
in an internet search (likely ending up at a Stack Overflow answer - which
we could create in advance of the 3.4.x release that includes this change).

That answer can then explain the various reasons this error can come up,
like:
* following a Python 2 tutorial in Python 3
* running a Python 2 script in Python 3
* attempting to install a Python 2 only dependency on Python 3

And possible solutions like:
* if it's your own code, changing the print statements to be compatible
with both Python 2 & 3 by using string formatting and surrounding
parentheses (& print("") for blank lines)
* looking for a Python 3 tutorial instead
* using Python 2 instead of Python 3
* looking for an alternative package that runs on Python 3

It's never going to be possible to fit all that into an error message,
hence why I consider "can be looked up in an internet search more easily
than the generic 'invalid syntax' error message" to be the most important
characteristic of the custom error message. I don't expect the exact
wording to really matter all that much.
msg220707 - (view) Author: Roundup Robot (python-dev) Date: 2014-06-16 09:50
New changeset 2b8cd2bc2745 by Nick Coghlan in branch '3.4':
Issue #21669: Special case print & exec syntax errors
http://hg.python.org/cpython/rev/2b8cd2bc2745

New changeset 36057f357537 by Nick Coghlan in branch 'default':
Merge issue #21669 from 3.4
http://hg.python.org/cpython/rev/36057f357537
msg220708 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-06-16 09:52
I went ahead and committed the v2 patch, as I think it's already an improvement over the generic message. Any further significant tweaks to the heuristics or changes to the error messages can be handled in separate issues.
msg225657 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-08-22 11:00
For the record, I just posted the Q&A reference to Stack Overflow: https://stackoverflow.com/questions/25445439/what-does-syntaxerror-missing-parentheses-in-call-to-print-mean-in-python/
History
Date User Action Args
2014-08-22 11:00:35ncoghlansetmessages: + msg225657
2014-06-16 09:52:14ncoghlansetstatus: open -> closed
type: enhancement
messages: + msg220708

resolution: fixed
stage: resolved
2014-06-16 09:50:07python-devsetnosy: + python-dev
messages: + msg220707
2014-06-09 01:32:05ncoghlansetmessages: + msg220077
2014-06-08 19:32:33glyphsetmessages: + msg220048
2014-06-07 12:37:07ncoghlansetfiles: + issue21669_custom_error_messages_v2.diff

messages: + msg219932
2014-06-06 16:46:23gvanrossumsetmessages: + msg219892
2014-06-06 16:29:11gvanrossumsetmessages: + msg219890
2014-06-06 12:57:15ncoghlansetfiles: + issue21669_custom_error_messages.diff

messages: + msg219880
2014-06-05 23:14:09gvanrossumsetmessages: + msg219855
2014-06-05 23:08:22ncoghlansetmessages: + msg219854
2014-06-05 16:15:30gvanrossumsetmessages: + msg219822
2014-06-05 15:07:19ncoghlancreate