classification
Title: First line can be executed twice
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Arfrever, benjamin.peterson, ezio.melotti, lemburg, loewis, python-dev, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2013-09-07 13:11 by serhiy.storchaka, last changed 2014-09-05 07:26 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
double_first_line.py serhiy.storchaka, 2013-09-07 13:11 Sample script
source_encoding_second_line.patch serhiy.storchaka, 2014-01-03 22:05 review
source_encoding_second_line_2.patch serhiy.storchaka, 2014-01-04 08:37 review
Messages (13)
msg197156 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-09-07 13:11
When run in Python 3 followed script:

print('first')
#coding=iso8859-1
print('second')

it prints the 'first' string twice.

Besides this weird behavior I think PEP-0263 should specify that if a magic comment is second line in the file then first line should be a comment too.
msg197157 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-09-07 13:14
> I think PEP-0263 should specify that if a magic comment is
> second line in the file then first line should be a comment too.

+1
msg197614 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-09-13 17:24
I thought that that was the specification*: the encoding comment should be the first line unless it has to be the second line to allow for an executable comment. Executing a non-comment first line twice is definitely contrary to the intent of the PEP.

* I have not read it recently ;-).

When running that snippet from 3.3.2 Idle editor, 'first' is only printed once. I consider this to be correct and I would not want to change Idle to match the buggy behavior.
msg197617 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2013-09-13 17:31
On 13.09.2013 19:24, Terry J. Reedy wrote:
> 
> Terry J. Reedy added the comment:
> 
> I thought that that was the specification*: the encoding comment should be the first line unless it has to be the second line to allow for an executable comment. Executing a non-comment first line twice is definitely contrary to the intent of the PEP.
> 
> * I have not read it recently ;-).
> 
> When running that snippet from 3.3.2 Idle editor, 'first' is only printed once. I consider this to be correct and I would not want to change Idle to match the buggy behavior.

Agreed. The PEP never intended to allow executable first lines in
a Python code file.
msg197938 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-09-16 19:41
It also makes non-working a common idiom for running Python files on Windows. When add first line "@python -x", it will be interpreted by Python and will cause SyntaxError.
msg207250 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-03 22:05
Here is a patch which fixes this issue (and related issues).

* Encoding declaration now detected in second line only if first line is spaces-only (this is needed for support -x option) or comment-only (needed for she-bang).

* Fixed support for -x option.

* PyTokenizer_FromString() no longer check encoding declaration in second line if it was found in first line.

* Fixed encoding detection in the tokenize module, in IDLE, in 2to3 and in the findnocoding script.
msg207274 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-04 08:37
Oh, I had not compiled the code after last change, so first patch is wrong. Here is fixed patch.
msg207576 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-07 18:14
Benjamin, current version (without switch) looks more clean to me. Are you insist?
msg207662 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-01-08 04:42
Yeah, I'm okay now.
msg207747 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-09 16:42
New changeset 1bdcaf6c0eb5 by Serhiy Storchaka in branch '3.3':
Issue #18960: Fix bugs with Python source code encoding in the second line.
http://hg.python.org/cpython/rev/1bdcaf6c0eb5

New changeset 04c05e408cbd by Serhiy Storchaka in branch 'default':
Issue #18960: Fix bugs with Python source code encoding in the second line.
http://hg.python.org/cpython/rev/04c05e408cbd
msg207757 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-09 17:10
Now traceback test is failed.

http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.3/builds/1358/steps/test/logs/stdio
======================================================================
FAIL: test_encoded_file (test.test_traceback.SyntaxTracebackCases)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/python/3.3.langa-ubuntu/build/Lib/test/test_traceback.py", line 146, in test_encoded_file
    text, charset, 4)
  File "/opt/python/3.3.langa-ubuntu/build/Lib/test/test_traceback.py", line 129, in do_test
    stdout[1], lineno))
AssertionError: 'line 4' not found in '  File "@test_6376_tmp", line 3, in <module>' : Invalid line number: '  File "@test_6376_tmp", line 3, in <module>' instead of 4

----------------------------------------------------------------------
msg207765 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-09 18:14
New changeset 875a514671dd by Serhiy Storchaka in branch '3.3':
Do not reset the line number because we already set file position to correct
http://hg.python.org/cpython/rev/875a514671dd

New changeset 2af308f79727 by Serhiy Storchaka in branch 'default':
Do not reset the line number because we already set file position to correct
http://hg.python.org/cpython/rev/2af308f79727
msg226405 - (view) Author: Roundup Robot (python-dev) Date: 2014-09-05 07:26
New changeset dd1e21f17b1c by Serhiy Storchaka in branch '2.7':
Issue #22221: Backported fixes from Python 3 (issue #18960).
http://hg.python.org/cpython/rev/dd1e21f17b1c
History
Date User Action Args
2014-09-05 07:26:25python-devsetmessages: + msg226405
2014-01-10 19:44:15serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-01-09 18:14:16python-devsetmessages: + msg207765
2014-01-09 17:10:46serhiy.storchakasetmessages: + msg207757
2014-01-09 16:42:28python-devsetnosy: + python-dev
messages: + msg207747
2014-01-08 04:42:49benjamin.petersonsetmessages: + msg207662
2014-01-07 18:14:36serhiy.storchakasetmessages: + msg207576
2014-01-04 08:37:23serhiy.storchakasetfiles: + source_encoding_second_line_2.patch

messages: + msg207274
2014-01-03 22:05:54serhiy.storchakasetfiles: + source_encoding_second_line.patch
messages: + msg207250

assignee: serhiy.storchaka
keywords: + patch
stage: patch review
2013-09-16 19:41:55serhiy.storchakasetmessages: + msg197938
2013-09-16 18:57:17Arfreversetnosy: + Arfrever
2013-09-13 17:31:11lemburgsetnosy: + lemburg
messages: + msg197617
2013-09-13 17:24:28terry.reedysetnosy: + terry.reedy
messages: + msg197614
2013-09-07 13:35:14pitrousetnosy: + benjamin.peterson
2013-09-07 13:14:18ezio.melottisetnosy: + ezio.melotti
messages: + msg197157
2013-09-07 13:11:33serhiy.storchakacreate