classification
Title: Inconsistent SyntaxError for print
Type: behavior Stage: resolved
Components: Parser Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: CuriousLearner, ammar2, aroberge, brandtbucher, corona10, eric.smith, iritkatriel, lys.nikolaou, miss-islington, ncoghlan, nitishch, pablogsal, piyushhajare, serhiy.storchaka, terry.reedy, veky, xtreak
Priority: release blocker Keywords: patch

Created on 2018-06-30 17:42 by corona10, last changed 2021-08-10 19:18 by terry.reedy. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27389 merged pablogsal, 2021-07-27 10:14
PR 27390 closed ammar2, 2021-07-27 16:03
PR 27391 merged pablogsal, 2021-07-27 16:21
PR 27392 merged pablogsal, 2021-07-27 16:33
PR 27393 merged miss-islington, 2021-07-27 20:31
PR 27521 merged pablogsal, 2021-07-31 23:15
PR 27522 merged miss-islington, 2021-08-01 01:11
Messages (33)
msg320797 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2018-06-30 17:42
Python 3.8.0a0 (heads/master-dirty:0cdf5f4289, Jul  1 2018, 02:30:31)
[Clang 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print "hi"
  File "<stdin>", line 1
    print "hi"
             ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("hi")?
>>> def add(a,b):
...     return a + b
...
>>> print add(3,5)
  File "<stdin>", line 1
    print add(3,5)
            ^
SyntaxError: invalid syntax

IMHO, an error message should be 'SyntaxError: Missing parentheses in call to 'print'. Did you mean print(add(3,5))?' but it is not.
msg320824 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-01 06:24
Related issue that introduced the error message if you would like to take a look at the implementation : https://bugs.python.org/issue30597

Thanks
msg320828 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2018-07-01 08:03
@xtreak

Thanks, I have interest with this issue :)
I will take a look at the implementation

Thanks!
msg320834 - (view) Author: Piyush Hajare (piyushhajare) Date: 2018-07-01 11:24
I'm interested to solve this issue
msg320836 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-01 13:33
I took an initial stab at this and there is a comment where if there is an open paren then to use the normal error message. Additional context on the issue which lists false positive cases where the change was introduced https://bugs.python.org/issue21669. I removed the check where the left paren count is not -1 in case of a function call and it seems to work. Of course, there are cases to handle as mentioned in the linked issue and I tried some of the cases like "foo".toupper().tolower() and the results as below : 

Patch : 

diff --git a/Objects/exceptions.c b/Objects/exceptions.c
index bb50c1c..7e616cf 100644
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -2966,10 +2966,7 @@ _report_missing_parentheses(PySyntaxErrorObject *self)
     if (left_paren_index < -1) {
         return -1;
     }
-    if (left_paren_index != -1) {
-        /* Use default error message for any line with an opening paren */
-        return 0;
-    }
+
     /* Handle the simple statement case */
     legacy_check_result = _check_for_legacy_statements(self, 0);
     if (legacy_check_result < 0) {

Cases :

➜  cpython git:(master) ✗ cat foo_print.py
class Foo:
    pass

print Foo().header(a=foo(1))
➜  cpython git:(master) ✗ ./python foo_print.py
  File "foo_print.py", line 4
    print Foo().header(a=foo(1))
            ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(Foo().header(a=foo(1)))?

➜  cpython git:(master) ✗ cat foo_print.py
print "a".toupper().tolower()
➜  cpython git:(master) ✗ ./python foo_print.py
  File "foo_print.py", line 1
    print "a".toupper().tolower()
            ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("a".toupper().tolower())?


Linked cases in the patch : 

➜  cpython git:(master) ✗ cat foo_print.py
print (foo.)
➜  cpython git:(master) ✗ ./python foo_print.py
  File "foo_print.py", line 1
    print (foo.)
               ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print((foo.))?


Would like to link to https://hackmd.io/s/ByMHBMjFe#08-Debugging-Python-Objects . As a beginner the tutorial was helpful and I had to set a break point on Objects/exceptions.c:1323 where SyntaxError_init is called and then execute line by line where the current code returns the left_paren_index which you can set as `set left_paren_index = -1` to skip the branch and then continue the program to print the above error messages. IMO I think it was an explicit decision with this change being done 4 years ago with the false positive cases and tests listed but I am curious if there are any improvements in this area to be done.


Thanks
msg320838 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2018-07-01 13:55
Thanks,

my easy patch is worked by pre-checking left paren index. but some edge case which I don't expect should be considered.

     if (left_paren_index != -1) {
-        /* Use default error message for any line with an opening paren */
-        return 0;
+	int found_white_space = 0;
+	int found_other_char = 0;
+	Py_ssize_t pos = 0;
+	while(pos != left_paren_index) {
+	    int kind = PyUnicode_KIND(self->text);
+	    void *data = PyUnicode_DATA(self->text);
+            Py_UCS4 ch = PyUnicode_READ(kind, data, pos);
+	    if (Py_UNICODE_ISSPACE(ch)) {
+                found_white_space = 1;
+	    }
+	    if (!Py_UNICODE_ISSPACE(ch) && found_white_space == 1) {
+                found_other_char = 1;
+	    }
+	    pos++;
+	}
+
+	if (found_other_char == 0) {
+	    return 0;
+	}
     }


Type "help", "copyright", "credits" or "license" for more information.
>>> print (foo.)
  File "<stdin>", line 1
    print (foo.)
               ^
SyntaxError: invalid syntax

>>> print "a".toupper().tolower()
  File "<stdin>", line 1
    print "a".toupper().tolower()
            ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("a".toupper().tolower())?

SyntaxError: Missing parentheses in call to 'print'. Did you mean print(Foo().header(a=foo(1)))?
msg320839 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2018-07-01 13:56
>>> class Foo:
...     pass
...
>>> print Foo().header(a=foo(1))
  File "<stdin>", line 1
    print Foo().header(a=foo(1))
            ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(Foo().header(a=foo(1)))?
msg320840 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2018-07-01 14:04
And here's my work in progress patch.

https://github.com/corona10/cpython/commit/133825346fd60e518e9ab5830a0755250c8c3e79
msg320851 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-02 04:49
@corona10 You might want to discuss this at https://mail.python.org/pipermail/python-ideas/ so that you get some initial feedback on the work and it's acceptance to the core.

Thanks
msg321196 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-07-07 01:20
Eric, Nick, Serhiy: this issue proposes to extend the print message patch of #30597 to cover more cases.  On the face of it, this seems sensible, but I have no read the original discussion or the current and proposed patches.
msg321212 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-07 09:04
See also #32685.
msg387153 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-02-17 13:15
Still the same in 3.10:

Python 3.10.0a5+ (heads/master:bf2e7e55d7, Feb 11 2021, 23:09:25) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print 3
  File "<stdin>", line 1
    print 3
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(3)?
>>> def f(x): return x
...
>>> print f(3)
  File "<stdin>", line 1
    print f(3)
          ^
SyntaxError: invalid syntax
msg387155 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-02-17 13:36
Would it be too much if add a Python 2 rule for print statement in grammar to produce better error message?

invalid_print_stmt:
    | 'print' ( test (',' test)* [','] ] |
                '>>' test [ (',' test)+ [','] ) {
      RAISE_INVALID_PRINT_STATEMENT() }
msg387158 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-02-17 14:14
> Would it be too much if add a Python 2 rule for print statement in grammar to produce better error message?

Please, go ahead. I think it makes sense and with our latest change in the parser, such new error message will have no impact on performance whatsoever. Please, at me as a reviewer :)
msg387289 - (view) Author: Vedran Čačić (veky) * Date: 2021-02-19 06:52
Aren't we overthinking this? Python 2 is a dead language. It has reached end of life more than a year ago (and was scheduled to do so in 2015). Why are we still trying to accomodate something that stopped being relevant a long time ago?
msg387290 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2021-02-19 06:57
It's still one of the most common beginner mistakes, personally I think the trade-off in complexity at least for the grammar level fix is worth it here.
msg387291 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-02-19 07:43
'Consistency' is in the eye of the beholder in that it is relative to some ideal.  'Inconsistent' has too much baggage as bad'.  I would prefer to call the current rule 'restricted' or 'limited' and judge any expansion on its own merits.

The arguments to print can take an unbounded amount of space.  So a general message "Did you mean print(<full argument list>)?" could also be indefinitely long.  I don't think this is a good idea.  Of course, the same applies to a single literal, especially multiline string literals.  But at least the latter are currently clipped to only the first line in the message.

>>> print '''first line
second line'''
SyntaxError: Missing parentheses in call to 'print'. Did you mean print('''first line)?

(The above would be better with trailing ... .) This abbreviation would be harder with multiple arguments.  The special message is for beginners.  They might print a literal or name without ()s more frequently.  I not sure that they need the reminder with every mistake.
msg387292 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-02-19 08:14
I agree with Terry’s point about printing long expressions in the error msg. Perhaps just the preamble would be useful though:

SyntaxError: Missing parentheses in call to 'print'. 

Instead of just

SyntaxError: invalid syntax.
msg387307 - (view) Author: Vedran Čačić (veky) * Date: 2021-02-19 11:53
> "It's still one of the most common beginner mistakes"

Do you have any data to back this up? I really think it's overblown.

On the other hand, if it really _is_ so, how about changing the language? It wouldn't be the first thing that was changed for Py3, and then changed back once people realized the old way was better.

It seems to me totally backwards to have a construct in the grammar, only to report it as an error. "I understand you, but you still have to use _this_ way to write what you want." I really think Python shouldn't be that kind of language.
msg387313 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-02-19 12:29
Let's step back a bit and focus on the issue at hand. The problem is the following:

* We **already** have a warning for the print statement without parens:

Python 3.9.1 (default, Dec 14 2020, 11:49:16)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print x
  File "<stdin>", line 1
    print x
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(x)?

This is achieved by inspecting the syntax error and checking some conditions, which I personally find it uglier than a resilient grammar rule.

* The question is if we want to make the rule more resilient or delete it whatsoever. The status quo doesn't seem like a good fit
msg387317 - (view) Author: Andre Roberge (aroberge) * Date: 2021-02-19 13:16
+1 to the idea of adding something to the grammar, and have a simple error message:

SyntaxError: Missing parentheses in call to 'print'.

in *all* cases, including the first one that prompted this bug report.

I write that even though I have created a third-party package based on the idea that beginners (and sometimes others...) might benefit from a more detailed error message (which is taken care of already by friendly-traceback).
msg398267 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-07-26 22:51
This case has changed recently, and not for the better:

>>> print f(3)
  File "<stdin>", line 1
    print f(3)
    ^^^^^^^^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?
msg398307 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-27 16:19
New changeset 6948964ecf94e858448dd28eea634317226d2913 by Pablo Galindo Salgado in branch 'main':
bpo-34013: Generalize the invalid legacy statement error message (GH-27389)
https://github.com/python/cpython/commit/6948964ecf94e858448dd28eea634317226d2913
msg398312 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-27 17:52
New changeset b977f8510e2ff4f11e3bda920722098a242fc8cc by Pablo Galindo Salgado in branch '3.10':
[3.10] bpo-34013: Generalize the invalid legacy statement error message (GH-27389). (GH-27391)
https://github.com/python/cpython/commit/b977f8510e2ff4f11e3bda920722098a242fc8cc
msg398316 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-27 20:30
New changeset ecc3c8e4216958d85385bf2467441c975128f26c by Pablo Galindo Salgado in branch 'main':
bpo-34013: Move the Python 2 hints from the exception constructor to the parser (GH-27392)
https://github.com/python/cpython/commit/ecc3c8e4216958d85385bf2467441c975128f26c
msg398318 - (view) Author: miss-islington (miss-islington) Date: 2021-07-27 21:19
New changeset 68e3dca0687c4c8e61ed98aed82f81049880c0ce by Miss Islington (bot) in branch '3.10':
bpo-34013: Move the Python 2 hints from the exception constructor to the parser (GH-27392)
https://github.com/python/cpython/commit/68e3dca0687c4c8e61ed98aed82f81049880c0ce
msg398652 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-07-31 17:25
Reopening as a release blocker.

It appears that this new rule is far too eager, and matches a much wider range of inputs than intended. Here is a case where it changes an IndentationError into a SyntaxError:

$ cat bug.py
print()
    boom

On 3.9 (correct):

$ ./python.exe --version
Python 3.9.6+
$ ./python.exe bug.py
  File "/Users/brandtbucher/Desktop/GitHub/cpython/bug.py", line 2
    boom
IndentationError: unexpected indent

On 3.10 (incorrect):

$ ./python.exe --version
Python 3.10.0b4+
$ ./python.exe bug.py
  File "/Users/brandtbucher/Desktop/GitHub/cpython/bug.py", line 1
    print()
    ^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

On 3.11 (incorrect):

$ ./python.exe --version
Python 3.11.0a0
$ ./python.exe bug.py
  File "/Users/brandtbucher/Desktop/GitHub/cpython/bug.py", line 1
    print()
    ^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

I recommend that this either be fixed or reverted before the RC.
msg398671 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-31 23:12
Is not about the eagerness, the problem is that it matches *first*, the parser never gets to the indentation error in the second phase.

For example, with:

print(3) $ 34

❯ ./python bug.py
  File "/home/pablogsal/github/python/main/bug.py", line 1
    print(3) $ 34
    ^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

The problem is that is matching the (3) as print + a number between parentheses. We just need to disallow to continue matching on the right
if it finds a '('.
msg398673 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-01 01:11
New changeset 208a7e957b812ad3b3733791845447677a704f3e by Pablo Galindo Salgado in branch 'main':
bpo-34013: Don't consider a grouped expression when reporting legacy print syntax errors (GH-27521)
https://github.com/python/cpython/commit/208a7e957b812ad3b3733791845447677a704f3e
msg398674 - (view) Author: miss-islington (miss-islington) Date: 2021-08-01 01:31
New changeset 35035bc35a9cb8617ab9fe9aac38aaf67c926aef by Miss Islington (bot) in branch '3.10':
bpo-34013: Don't consider a grouped expression when reporting legacy print syntax errors (GH-27521)
https://github.com/python/cpython/commit/35035bc35a9cb8617ab9fe9aac38aaf67c926aef
msg399317 - (view) Author: Andre Roberge (aroberge) * Date: 2021-08-10 12:41
Python 3.10.0rc1 ...

>>> print hello world!
  File "<stdin>", line 1
    print hello world!
          ^^^^^^^^^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?

The hint given is not exactly helpful ...

(This example was in a discussion on Twitter https://twitter.com/cfbolz/status/1425036431974715400 about a previous handling of this invalid syntax case where it was mentioned that pypy verifies that the suggestion it makes actually yields syntactically valid code.)
msg399323 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-10 13:15
If we change the priority of the error messages, which is unclear if is easily possible, the error will suggest that

print hello should be parenthesized as print (hello) Which if corrected will leave the "world" part out as a call followed to a name, and here the comma suggestion is a valid concern.

I understand the willingness to have a better error here, but I am not sure what we can improve here.
msg399354 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-08-10 19:18
I think that this was properly closed after the last fix.  There are multiple issues at play:

1. These parts of the Zen:
"Special cases aren't special enough to break the rules.
Although practicality beats purity.
...
In the face of ambiguity, refuse the temptation to guess."

2. The fact that a parser only reads and checks *python* syntax, while we humans have to make a special effort to avoid semantics.  The latter is especially true when the semantic is a special-case computing trope. Even if an identifier is semantically recognized as a function, its signature (call syntax) is specific to the function and is not part of python syntax.

3. The that with PEG parsing, there is no exact failure point. (Pablo just documented the PEG parser.  But, Pablo, where is it?  Not in HOWTO or index.)  This is especially true when there are multiple errors, as in Andre's example.

As long as we only had one special case hint, which depended on the semantic knowledge that 'print' is almost certainly meant to mean print with the builtin function with a known call syntax, it was fairly easy to get the hint right, and in 3.11 we still have

>>> print """jsfl sf
... sjflsfj l
... sjflsjf
... sjfs fj
... sjlsfjlsjg
... """
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

But we now have more hints, and "print Hello world!" is syntactically the same as "a b c!", which gives the same error message.  Even if the first identifier is recognized as a function, and parenthesis are added, the result would be syntactically identical to "print(b c!)".  At this  point, it is syntactically ambiguous whether the fixed argument should be a string "'b c!'" or comma list "b, c".  Both are possible for print, but not all functions.  The 'correct' fix on only 'obvious' when we add knowledge of the semantics 'Hello, 'world', and '!' and the history of their concatenation, 'Hello world!' (but sometimes without '!' or capitalization, or with <name> replacing 'world'), in computing, and especially in the teaching of beginners in a particular language.
History
Date User Action Args
2021-08-10 19:18:12terry.reedysetmessages: + msg399354
2021-08-10 13:15:04pablogsalsetmessages: + msg399323
2021-08-10 12:41:18arobergesetmessages: + msg399317
2021-08-01 01:31:51miss-islingtonsetmessages: + msg398674
2021-08-01 01:11:49pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-08-01 01:11:17pablogsalsetmessages: + msg398673
2021-08-01 01:11:14miss-islingtonsetpull_requests: + pull_request26037
2021-07-31 23:15:58pablogsalsetstage: needs patch -> patch review
pull_requests: + pull_request26036
2021-07-31 23:12:37pablogsalsetmessages: + msg398671
2021-07-31 17:25:32brandtbuchersetstatus: closed -> open
priority: normal -> release blocker
type: enhancement -> behavior

components: + Parser, - Interpreter Core

nosy: + brandtbucher
messages: + msg398652
resolution: fixed -> (no value)
stage: resolved -> needs patch
2021-07-27 21:20:34pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-07-27 21:19:55miss-islingtonsetmessages: + msg398318
2021-07-27 20:31:04miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25926
2021-07-27 20:30:44pablogsalsetmessages: + msg398316
2021-07-27 17:52:50pablogsalsetmessages: + msg398312
2021-07-27 16:33:16pablogsalsetpull_requests: + pull_request25925
2021-07-27 16:21:46pablogsalsetpull_requests: + pull_request25924
2021-07-27 16:19:31pablogsalsetmessages: + msg398307
2021-07-27 16:03:10ammar2setpull_requests: + pull_request25923
2021-07-27 10:14:50pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25921
2021-07-26 22:51:34iritkatrielsetmessages: + msg398267
2021-02-20 18:04:02lys.nikolaousetnosy: + lys.nikolaou
2021-02-19 13:16:38arobergesetnosy: + aroberge
messages: + msg387317
2021-02-19 12:29:13pablogsalsetmessages: + msg387313
2021-02-19 11:53:47vekysetmessages: + msg387307
2021-02-19 08:14:28iritkatrielsetmessages: + msg387292
2021-02-19 07:43:52terry.reedysetmessages: + msg387291
2021-02-19 06:57:48ammar2setnosy: + ammar2
messages: + msg387290
2021-02-19 06:52:54vekysetnosy: + veky
messages: + msg387289
2021-02-17 14:14:57pablogsalsetmessages: + msg387158
2021-02-17 13:36:51serhiy.storchakasetnosy: + pablogsal
messages: + msg387155
2021-02-17 13:15:54iritkatrielsetnosy: + iritkatriel

messages: + msg387153
versions: + Python 3.10, - Python 3.8
2018-07-07 09:04:56serhiy.storchakasetnosy: + CuriousLearner, nitishch
messages: + msg321212
2018-07-07 01:20:51terry.reedysetnosy: + eric.smith, terry.reedy, serhiy.storchaka, ncoghlan
messages: + msg321196
2018-07-02 04:49:41xtreaksetmessages: + msg320851
2018-07-01 14:04:41corona10setmessages: + msg320840
2018-07-01 13:56:12corona10setmessages: + msg320839
2018-07-01 13:55:45corona10setmessages: + msg320838
2018-07-01 13:33:43xtreaksetmessages: + msg320836
2018-07-01 11:24:22piyushhajaresetnosy: + piyushhajare
messages: + msg320834
2018-07-01 08:03:04corona10setmessages: + msg320828
2018-07-01 06:24:57xtreaksetnosy: + xtreak
messages: + msg320824
2018-06-30 17:43:31corona10settype: enhancement
components: + Interpreter Core
2018-06-30 17:42:58corona10create