classification
Title: Make compile('1\n2\n', '', 'single') raise an exception instead of silently truncating?
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Devin Jeanpierre, benjamin.peterson, eric.araujo, georg.brandl, meador.inge, python-dev, terry.reedy
Priority: normal Keywords: patch

Created on 2011-08-07 01:39 by Devin Jeanpierre, last changed 2012-01-19 07:12 by meador.inge. This issue is now closed.

Files
File name Uploaded Description Edit
issue12705-0.patch meador.inge, 2012-01-17 15:52 Patch against tip (3.3.0a0) review
Messages (11)
msg141735 - (view) Author: Devin Jeanpierre (Devin Jeanpierre) * Date: 2011-08-07 01:39
(this is copy-pasted from http://mail.python.org/pipermail/python-ideas/2011-July/010787.html )

compile('1\n2\n', '','single') == compile('1\n', '','single').

That is, it ignores the second statement ('2\n'),
without offering a way for the caller to detect this.

Considering that 'single' is primarily used to emulate the behaviour
of the Python interpreter, most of the time, giving it multiple
statements is an impossibility, and so that case doesn't matter and
could raise an exception without affecting existing code. For example,
the code module meets this description, as do debuggers and such.

However, in cases where it _is_ possible to give the compiler multiple
statements, the user should be warned that his input isn't valid,
somehow. For example, the following doctest will mysteriously fail,
because it was written incorrectly (doctest uses 'single'):

   >>> import sys
   ... sys.stdout.write('foo\n')
   foo

This is because the second statement in the doctest was silently
discarded by compile(). It might not always be clear to users how to
fix this, and I think this kind of non-obvious error would exist in
any use of 'single' that can in theory involve multiple statements,
through user error or program bug. So I'd appreciate it if compile()
raised an exception in this case. Perhaps SyntaxError or ValueError.
msg141977 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-08-12 17:46
Pending an argument against, I agree with the change.
I think SyntaxError would be best. ValueError (etc) is for runtime (though this is compile during runtime).

What would you have for the error message? My first idea is
"Cannot compile multiple statements as a single statement."
This should be clear enough when calling compile(...'single').
It should at least give a hint in the doctest case.
msg151319 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-01-16 04:56
IMO this is a bug and should be fixed in stable versions too.
msg151373 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-01-16 15:25
I think an exception is fine, but it should only happen in 3.3.
msg151461 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2012-01-17 15:52
The attached patch fixes this be checking what is left in the input
buffer after parsing.  Anything but trailing whitespace and comments
triggers the exception.

Ignoring trailing whitespace and comments makes sense, but it is also 
somewhat required.  Trailing comments are common in doctests:

    >>> doctest.testfile('test_doctest.txt', raise_on_error=True)
    ... # doctest: +ELLIPSIS

and trailing whitespace is used by the codeop heuristics.

Here is an example of the new exception:

>>> compile('1 + 2\n3 + 4\n', '','single')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "", line 1
    1 + 2
        ^
SyntaxError: multiple statements found while compiling a single statement

Tested on Fedora 15; no regressions.
msg151475 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-01-17 16:56
I don’t understand why some two-liners are allowed (like "class X:\n pass").  The doc says “a single interactive statement”.
msg151481 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2012-01-17 19:20
On Tue, Jan 17, 2012 at 10:56 AM, Éric Araujo <report@bugs.python.org> wrote:

> I don’t understand why some two-liners are allowed (like "class X:\n pass").  The doc says “a single interactive statement”.

Because a single statement can be multiple lines (as is the case for
compound statements).  Look at the grammar for a single input
statement
(interactive_input):
http://docs.python.org/dev/reference/toplevel_components.html#interactive-input.
 This issue is really about multiple statements
and not multiple lines.
msg151482 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-01-17 19:25
Because a class statement is one statement (it is a compound statement, but still just one).  The docs don't talk about lines :)

A "single interactive statement" is what you can enter at the interactive prompt in one line, or more than one line with only secondary prompts from the second line on.
msg151500 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-01-18 00:06
Thank you both for fixing my incomplete understanding.

Meador, unless you want a review from another core dev, I think you can just go ahead.
msg151609 - (view) Author: Roundup Robot (python-dev) Date: 2012-01-19 07:11
New changeset 2bd7f40108b4 by Meador Inge in branch 'default':
Issue #12705: Raise SyntaxError when compiling multiple statements as single interactive statement
http://hg.python.org/cpython/rev/2bd7f40108b4
msg151610 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2012-01-19 07:12
Fixed in 3.3.
History
Date User Action Args
2012-01-19 07:12:14meador.ingesetstatus: open -> closed
resolution: fixed
messages: + msg151610

stage: patch review -> resolved
2012-01-19 07:11:11python-devsetnosy: + python-dev
messages: + msg151609
2012-01-18 00:06:11eric.araujosetmessages: + msg151500
2012-01-17 19:25:20georg.brandlsetnosy: + georg.brandl
messages: + msg151482
2012-01-17 19:20:33meador.ingesetmessages: + msg151481
2012-01-17 16:56:51eric.araujosetmessages: + msg151475
2012-01-17 15:52:07meador.ingesetfiles: + issue12705-0.patch
keywords: + patch
messages: + msg151461

stage: needs patch -> patch review
2012-01-16 18:00:09meador.ingesetnosy: + meador.inge
2012-01-16 15:25:13benjamin.petersonsetmessages: + msg151373
2012-01-16 12:23:25pitrousetnosy: + benjamin.peterson

stage: test needed -> needs patch
2012-01-16 04:56:42eric.araujosetnosy: + eric.araujo
messages: + msg151319
2011-08-12 17:46:22terry.reedysetnosy: + terry.reedy
messages: + msg141977

type: enhancement
stage: test needed
2011-08-07 01:39:51Devin Jeanpierrecreate