classification
Title: null source chars handled oddly by tokenize
Type: behavior Stage: needs patch
Components: Interpreter Core Versions: Python 3.2, Python 3.1, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Alexandre.Zani, ajaksu2, georg.brandl, rcharney
Priority: normal Keywords:

Created on 2005-01-20 07:35 by rcharney, last changed 2012-05-21 03:25 by Alexandre.Zani.

Files
File name Uploaded Description Edit
invalidChars.py rcharney, 2005-01-20 07:35
Messages (3)
msg24004 - (view) Author: Reginald B. Charney (rcharney) Date: 2005-01-20 07:35
When null characters appear in the source, outside
literals, tokenize seems to either: skip the null
character and the next two following characters; or
ignore the remainder of the line, including the newline
character.

(To see the invalid characters, use vim, or an editor
that displays control characters when needed.)
msg24005 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-01-10 23:07
Logged In: YES 
user_id=1188172

Confirmed with current SVN heads.
msg82112 - (view) Author: Daniel Diniz (ajaksu2) Date: 2009-02-14 19:09
Confirmed on trunk at rev69546.
History
Date User Action Args
2012-05-21 03:25:25Alexandre.Zanisetnosy: + Alexandre.Zani
2010-08-19 18:52:07BreamoreBoysettitle: null source chars handled oddly -> null source chars handled oddly by tokenize
stage: test needed -> needs patch
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6
2009-02-14 19:09:46ajaksu2setnosy: + ajaksu2
stage: test needed
type: behavior
messages: + msg82112
versions: + Python 2.6
2005-01-20 07:35:04rcharneycreate