classification
Title: Python 3.3.4: SyntaxError with correct source code encoding # -*- coding: latin-1 -*-
Type: compile error Stage: resolved
Components: Interpreter Core, Windows Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, OPi, georg.brandl, haypo, koobs, larry, loewis, miwa, python-dev, r.david.murray, serhiy.storchaka
Priority: release blocker Keywords: patch

Created on 2014-02-22 19:20 by OPi, last changed 2014-03-01 21:34 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
test.py OPi, 2014-02-22 19:20
test1.py miwa, 2014-02-24 07:12
issue20731.patch serhiy.storchaka, 2014-02-24 09:08 review
Messages (15)
msg211942 - (view) Author: (OPi) Date: 2014-02-22 19:20
The test program:
#!/usr/bin/env python
# -*- coding: latin-1 -*-

print('test')

is correct in Python 3.3.3, but cause this error with Python 3.3.4:
  File "./test.py", line 3
    -
    ^
SyntaxError: invalid syntax

I use Windows 7, with bash (of Cygwin) and
Python 3.3.4 (v3.3.4:7ff62415e426, Feb 10 2014, 18:13:51) [MSC v.1600 64 bit (AMD64)] on win32
msg211961 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-02-22 21:43
For whatever it's worth, it works fine for me on 3.3 tip on unix.  The fact that it says line 3 and shows just a - is suspicious.
msg212038 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-02-23 22:50
I can reproduce the problem. I started debugging it; the issue is that, after reading the two comment lines (and detecting the source encoding), fileio_readinto gets called, which calls read(), and read() returns 17 (bytes read), with are

"-\n\nprint('test')"

So for some reason, the - is not consumed yet, even though it was previously read out of the file descriptor.
msg212046 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-02-23 23:54
Oops, I missed the fact that it was Benjamin that added Serhiy...Benjamin can add himself if he wants :)
msg212063 - (view) Author: Musashi Tamura (miwa) Date: 2014-02-24 07:12
You can avoid this by changing end of line to CRLF. (I don't know why)

I attach a simpler test case.
msg212068 - (view) Author: (OPi) Date: 2014-02-24 08:25
All works fine with UTF-8.
All works fine with latin-1 (or other) and Windows end of line.

Maybe this issue is important, "First line can be executed twice":
http://bugs.python.org/issue18960
msg212073 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-24 09:08
I can't test on Windows but may be this patch fixes the issue.
msg212074 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-02-24 09:08
Just in time for 3.3.5...
msg212437 - (view) Author: Roundup Robot (python-dev) Date: 2014-02-28 14:49
New changeset ade5e4922a54 by Martin v. Löwis in branch '3.3':
Issue #20731: Properly position in source code files even if they
http://hg.python.org/cpython/rev/ade5e4922a54
msg212438 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-02-28 14:52
Thanks for the patch, this is applied to 3.3 and 3.4.
msg212445 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-02-28 15:17
Regressions on buildbots, examples:

http://buildbot.python.org/all/builders/x86%20OpenIndiana%203.3/builds/1458
http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.3/builds/1546
http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.3/builds/464
etc.
msg212492 - (view) Author: koobs (koobs) Date: 2014-03-01 08:40
Regressions on FreeBSD buildbots (test_coding)

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%203.3/builds/626/steps/test/logs/stdio
msg212504 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-03-01 15:15
This needs to be fixed for 3.3.5rc2.
msg212505 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-01 15:34
New changeset dcf4fbf446ca by Benjamin Peterson in branch '3.3':
fix test on debug builds (closes #20731)
http://hg.python.org/cpython/rev/dcf4fbf446ca
msg212520 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-03-01 21:34
Thanks for fixing it.
History
Date User Action Args
2014-03-29 14:19:50Arfreverlinkissue21086 superseder
2014-03-01 21:34:15loewissetmessages: + msg212520
2014-03-01 15:34:34python-devsetstatus: open -> closed
resolution: fixed
messages: + msg212505

stage: patch review -> resolved
2014-03-01 15:15:24georg.brandlsetmessages: + msg212504
2014-03-01 10:27:10Arfreversetstatus: closed -> open
resolution: fixed -> (no value)
2014-03-01 08:40:35koobssetnosy: + koobs
messages: + msg212492
2014-02-28 15:17:31hayposetnosy: + haypo
messages: + msg212445
2014-02-28 14:52:06loewissetstatus: open -> closed
resolution: fixed
messages: + msg212438
2014-02-28 14:49:39python-devsetnosy: + python-dev
messages: + msg212437
2014-02-24 09:08:47georg.brandlsetmessages: + msg212074
2014-02-24 09:08:00serhiy.storchakasetfiles: + issue20731.patch

versions: + Python 3.4
keywords: + patch
nosy: + larry

messages: + msg212073
stage: patch review
2014-02-24 09:01:19loewissetpriority: normal -> release blocker
nosy: + georg.brandl
2014-02-24 08:25:10OPisetmessages: + msg212068
2014-02-24 07:58:43serhiy.storchakasetcomponents: + Windows
2014-02-24 07:12:30miwasetfiles: + test1.py
nosy: + miwa
messages: + msg212063

2014-02-24 04:56:46Arfreversetnosy: + Arfrever
2014-02-23 23:54:40r.david.murraysetnosy: - benjamin.peterson
messages: + msg212046
2014-02-23 23:53:29r.david.murraysetnosy: + benjamin.peterson
2014-02-23 22:59:15benjamin.petersonsetnosy: + serhiy.storchaka
2014-02-23 22:50:54loewissetnosy: + loewis
messages: + msg212038
2014-02-22 21:43:49r.david.murraysetnosy: + r.david.murray
messages: + msg211961
2014-02-22 19:20:15OPicreate