This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: python -m imports non-ASCII .py file without encoding declaration
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, eryksun, iritkatriel, jwilk, loewis, ncoghlan, r.david.murray, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2013-12-10 11:04 by jwilk, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
test.py jwilk, 2013-12-10 11:04
Messages (8)
msg205787 - (view) Author: Jakub Wilk (jwilk) Date: 2013-12-10 11:04
If you have a non-ASCII .py file without encoding declaration, then you can't normally import it:

$ python --version
Python 2.7.6

$ python -c 'import test'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "test.py", line 1
SyntaxError: Non-ASCII character '\xc2' in file test.py on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details


However, "python -m" happily imports such files:

$ python -m test
¡Hello world!
msg205820 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-12-10 14:56
Well, we aren't going to change 2.7 to have this case start throwing an error, since someone may be depending on it.  So I'm not sure there's anything to do here.
msg205824 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-10 15:16
There are several bugs in processing encoding declaration (issue18961, issue18873) and yet several were fixed last time. Perhaps this issue or issue19942 relate to them.
msg205865 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-12-10 22:49
Quite plausibly a bug in the pkgutil import system emulation. Does the same inconsistency appear in 3.3 with non-UTF8 bytes?
msg205878 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2013-12-11 02:33
3.x is afflicted.
msg407386 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-30 17:04
On 3.11 both are working (on a Mac):

cpython-1 % python -m tt  
¡Hello world!
cpython-1 % ./python.exe -c 'import tt'
¡Hello world!
msg407414 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-11-30 23:55
test.py is a UTF-8 file, which is the default source encoding in Python 3. It fails as expected if the test script is encoded differently, such as Latin-1, unless the source encoding is declared.
msg407419 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-12-01 00:45
I confirm: Python 3.10 works as expected.

Python 3.10 fails with the same SyntaxError using "python script.py" or "python -m script" if the script contains non-ASCII characters but is not encoded to UTF-8.

vstinner@apu$ python3 test.py
  File "/home/vstinner/test.py", line 1
    print('�Hello world!')
                         ^
SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0xa1 in position 0: invalid start byte

vstinner@apu$ python3 -m test
Traceback (most recent call last):
  (...)
  File "/home/vstinner/test.py", line 1
    print('�Hello world!')
                         ^
SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0xa1 in position 0: invalid start byte
History
Date User Action Args
2022-04-11 14:57:55adminsetgithub: 64140
2021-12-01 00:45:37vstinnersetmessages: + msg407419
2021-11-30 23:55:23eryksunsetstatus: open -> closed

nosy: + eryksun
messages: + msg407414

resolution: out of date
stage: resolved
2021-11-30 17:05:02iritkatrielsetcomponents: + Interpreter Core
2021-11-30 17:04:12iritkatrielsetnosy: + iritkatriel
messages: + msg407386
2020-03-06 20:47:02brett.cannonsetnosy: - brett.cannon
2013-12-11 02:33:44benjamin.petersonsetnosy: + benjamin.peterson

messages: + msg205878
versions: + Python 3.3, Python 3.4, - Python 2.7
2013-12-10 22:49:46ncoghlansetmessages: + msg205865
2013-12-10 15:16:16serhiy.storchakasetmessages: + msg205824
2013-12-10 14:56:00r.david.murraysetnosy: + r.david.murray, ncoghlan
messages: + msg205820
2013-12-10 11:58:43serhiy.storchakasetnosy: + loewis, serhiy.storchaka
type: behavior
2013-12-10 11:06:32vstinnersetnosy: + brett.cannon, vstinner
2013-12-10 11:04:57jwilkcreate