Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(46944)

Unified Diff: Lib/tokenize.py

Issue 18873: "Encoding" detected in non-comment lines
Patch Set: Created 6 years ago
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « Lib/test/test_tokenize.py ('k') | Tools/scripts/findnocoding.py » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
--- a/Lib/tokenize.py Thu Sep 05 18:02:31 2013 +0300
+++ b/Lib/tokenize.py Sat Sep 07 17:49:25 2013 +0300
@@ -31,7 +31,7 @@
from codecs import lookup, BOM_UTF8
import collections
from io import TextIOWrapper
-cookie_re = re.compile("coding[:=]\s*([-\w.]+)")
+cookie_re = re.compile(r'^[ \t\f]*#.*coding[:=][ \t]*([-\w.]+)', re.ASCII)
import token
__all__ = token.__all__ + ["COMMENT", "tokenize", "detect_encoding",
@@ -372,10 +372,10 @@
msg = '{} for {!r}'.format(msg, filename)
raise SyntaxError(msg)
- matches = cookie_re.findall(line_string)
- if not matches:
+ match = cookie_re.match(line_string)
+ if not match:
return None
- encoding = _get_normal_name(matches[0])
+ encoding = _get_normal_name(match.group(1))
try:
codec = lookup(encoding)
except LookupError:
« no previous file with comments | « Lib/test/test_tokenize.py ('k') | Tools/scripts/findnocoding.py » ('j') | no next file with comments »

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+