➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2008-10-02 23:00 by vstinner, last changed 2022-04-11 14:56 by admin. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| tokenize_detect_encoding.patch | vstinner, 2008-10-02 23:00 | |||
| tokenize_detect_encoding-2.patch | vstinner, 2008-12-11 23:20 | |||
| Messages (4) | |||
|---|---|---|---|
| msg74205 - (view) | Author: STINNER Victor (vstinner) *  |
Date: 2008-10-02 23:00 | |
tokenize.detect_encoding() raises a LookupError() if the charset is unknown whereas Python raises a SyntaxError. So this patch mimics Python behaviour for tokenize module. Extra: reuse BOM_UTF8 from the codecs module. |
|||
| msg74217 - (view) | Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2008-10-02 23:49 | |
This patch seems good to me, it fixes the following test: Index: Lib/test/test_tokenize.py =================================================================== --- Lib/test/test_tokenize.py (revision 66701) +++ Lib/test/test_tokenize.py (working copy) @@ -795,6 +795,8 @@ self.assertEquals(encoding, 'utf-8') self.assertEquals(consumed_lines, []) + readline = self.get_readline((b'# coding: bad\n',)) + self.assertRaises(SyntaxError, detect_encoding, readline) class TestTokenize(TestCase): |
|||
| msg77642 - (view) | Author: STINNER Victor (vstinner) * |
Date: 2008-12-11 23:20 | |
New version of the patch: - remove utf8_bom (was already replaced by codecs.BOM_UTF8) - include the regression test from amaury.forgeotdarc Can anyone review the new patch (which is very similar to the first one) and commit it? |
|||
| msg77652 - (view) | Author: Benjamin Peterson (benjamin.peterson) * |
Date: 2008-12-12 01:25 | |
Fixed in r67711. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:56:39 | admin | set | github: 48271 |
| 2008-12-12 01:25:48 | benjamin.peterson | set | status: open -> closed keywords: patch, patch, needs review resolution: fixed messages: + msg77652 nosy: + benjamin.peterson |
| 2008-12-11 23:21:00 | vstinner | set | keywords:
patch, patch, needs review files: + tokenize_detect_encoding-2.patch messages: + msg77642 |
| 2008-10-02 23:49:03 | amaury.forgeotdarc | set | keywords:
+ needs review nosy: + amaury.forgeotdarc messages: + msg74217 |
| 2008-10-02 23:00:32 | vstinner | create | |