Created on 2018-08-27 06:47 by monson, last changed 2018-09-15 17:53 by miss-islington. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 8950 | merged | monson, 2018-08-27 06:54 | |
| PR 9333 | merged | miss-islington, 2018-09-15 17:32 | |
| Messages (3) | |||
|---|---|---|---|
| msg324148 - (view) | Author: monson (monson) * | Date: 2018-08-27 06:47 | |
Python 3.0 introduces additional characters from outside the ASCII range (see PEP 3131). see https://docs.python.org/3/reference/lexical_analysis.html#identifiers But lib2to3 can't tokenize them corretly. ``` $ echo '中 = 1' | python3.7 -m lib2to3.pgen2.tokenize 1,0-1,1: ERRORTOKEN '中' 1,2-1,3: OP '=' 1,4-1,5: NUMBER '1' 1,5-1,6: NEWLINE '\n' 2,0-2,0: ENDMARKER '' ``` '中' should be tokenized as NAME instead of ERRORTOKEN. |
|||
| msg325451 - (view) | Author: Benjamin Peterson (benjamin.peterson) *  |
Date: 2018-09-15 17:32 | |
New changeset 10a428b64b3f224e2ccd40ff2afb141b9b3425b1 by Benjamin Peterson (Monson Shao) in branch 'master': closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) https://github.com/python/cpython/commit/10a428b64b3f224e2ccd40ff2afb141b9b3425b1 |
|||
| msg325452 - (view) | Author: miss-islington (miss-islington) | Date: 2018-09-15 17:53 | |
New changeset 51dbae867e82014f9af89662977e4981463c51e8 by Miss Islington (bot) in branch '3.7': closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) https://github.com/python/cpython/commit/51dbae867e82014f9af89662977e4981463c51e8 |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2018-09-15 17:53:02 | miss-islington | set | nosy:
+ miss-islington messages: + msg325452 |
| 2018-09-15 17:32:49 | miss-islington | set | pull_requests: + pull_request8756 |
| 2018-09-15 17:32:33 | benjamin.peterson | set | status: open -> closed nosy: + benjamin.peterson messages: + msg325451 resolution: fixed stage: patch review -> resolved |
| 2018-08-27 08:39:07 | monson | set | versions: + Python 3.8, - Python 3.7 |
| 2018-08-27 06:54:19 | monson | set | keywords:
+ patch stage: patch review pull_requests: + pull_request8426 |
| 2018-08-27 06:47:47 | monson | create | |