This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author terry.reedy
Recipients BTaskaya, lys.nikolaou, pablogsal, terry.reedy, tusharsadhwani
Date 2022-01-29.05:52:02
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1643435522.57.0.816086419574.issue46521@roundup.psfhosted.org>
In-reply-to
Content
Tushar, I am the principle IDLE maintainer and have dealt with similar interactive compile problems on and off for a few years. Code module uses codeop._maybe_compile, which in turn uses batch-mode compile().

To review: Code can be in 3 states: legal, illegal (there is an positive error that must be fixed), and incomplete (might become legal with more input).  Batch mode compile has all the code it is going to get, so it raises an exception for both bad and incomplete code.

In command-line languages, '\n' definitely signals end-of-command. Even in interactive mode, incomplete commands get an error message. The user must retype or recall and add more.

Python is a multiline and compound statement language.  Newline may instead signal the end of a compound statement header or the end of a nested statement, or even just be present for visual formatting.  Being able to continue incomplete statements after newline is essential.

In interactive mode, the interpreter looks at code after each newline and differentiates between unrecoverable and merely incomplete errors.  In the latter case, it sends a prompt to enter more and then reads more.

codeop._maybe compile attempts to simulate interactive mode and make a trinary decision (returning None for 'incomplete') using batch-mode binary compile().  A hack using repeated compiles, a warning filter, and a helper function classifies code that failed the initial compile.  I suspect that a) it has never been perfect and b) it cannot be (see experiment below).

The issue here is that _maybe_compile returns None instead of passing on the compile syntax error.  Some debug prints would reveal exactly why.

An alternate approach might be to compile just once and use the error message and marked error range to split.  But that would require different message-range pairs for the different cases.  Compile does not seem to give this to us.  For the current case, 3.11.a04 compile (and ast.parse) give the same response to both bad and incomplete lines.

>>> compile("a b '''", '', 'single')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "", line 1
    a b '''
        ^
SyntaxError: unterminated triple-quoted string literal (detected at line 1)
>>> compile("s='''", '', 'single')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "", line 1
    s='''
      ^
SyntaxError: unterminated triple-quoted string literal (detected at line  1

But the REPL somehow treats the two lines differently when directly entered.

>>> s = '''
...
... '''
>>> a b '''
  File "<stdin>", line 1
    a b '''
        ^
SyntaxError: unterminated triple-quoted string literal (detected at line 1)

Pablo, is there any possibility that the internal REPL parser could be wrapped, exposed to Python, and called with fake stdin/out/err objects?

Or if one executes 'python -i -c "start code"', what is required of the standard streams for '-i' to be effective and actually shift into interactive mode, reading from and writing to those streams?  Just claim to be a tty?  It is any different than for reading responses to "input('prompt')"?
History
Date User Action Args
2022-01-29 05:52:02terry.reedysetrecipients: + terry.reedy, lys.nikolaou, pablogsal, BTaskaya, tusharsadhwani
2022-01-29 05:52:02terry.reedysetmessageid: <1643435522.57.0.816086419574.issue46521@roundup.psfhosted.org>
2022-01-29 05:52:02terry.reedylinkissue46521 messages
2022-01-29 05:52:02terry.reedycreate