Title: null source chars handled oddly by tokenize
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.1, Python 3.2, Python 2.7
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: Nosy List: Alexandre.Zani, ajaksu2, gvanrossum, rcharney
Priority: normal Keywords:

Created on 2005-01-20 07:35 by rcharney, last changed 2021-06-17 15:19 by gvanrossum. This issue is now closed.

File name Uploaded Description Edit rcharney, 2005-01-20 07:35
Messages (4)
msg24004 - (view) Author: Reginald B. Charney (rcharney) Date: 2005-01-20 07:35
When null characters appear in the source, outside
literals, tokenize seems to either: skip the null
character and the next two following characters; or
ignore the remainder of the line, including the newline

(To see the invalid characters, use vim, or an editor
that displays control characters when needed.)
msg24005 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-01-10 23:07
Logged In: YES 

Confirmed with current SVN heads.
msg82112 - (view) Author: Daniel Diniz (ajaksu2) (Python triager) Date: 2009-02-14 19:09
Confirmed on trunk at rev69546.
msg396005 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-06-17 15:19
Closing in favor of issue20115.
Date User Action Args
2021-06-17 15:19:21gvanrossumsetstatus: open -> closed

nosy: + gvanrossum
messages: + msg396005

resolution: duplicate
stage: needs patch -> resolved
2020-09-19 19:04:20georg.brandlsetnosy: - georg.brandl
2012-05-21 03:25:25Alexandre.Zanisetnosy: + Alexandre.Zani
2010-08-19 18:52:07BreamoreBoysettitle: null source chars handled oddly -> null source chars handled oddly by tokenize
stage: test needed -> needs patch
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6
2009-02-14 19:09:46ajaksu2setnosy: + ajaksu2
stage: test needed
type: behavior
messages: + msg82112
versions: + Python 2.6
2005-01-20 07:35:04rcharneycreate