classification
Title: PEP 414 tokenizing hook does not preserve tabs
Type: behavior Stage:
Components: Versions: Python 3.3
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, aronacher, vinay.sajip
Priority: normal Keywords:

Created on 2012-05-03 15:43 by vinay.sajip, last changed 2014-07-11 21:29 by aronacher. This issue is now closed.

Messages (6)
msg159865 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-05-03 15:43
Tabs in Python source are a no-no for me and lots of other people, but some people do use them. The tokenizer seems to not preserve tabs in its output. There are some tabs in the Django source, for example tests/regressiontests/localflavor/pl/tests.py.

N.B. I posted this to your GitHub repo where you published the hook.
msg222684 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-10 18:23
PEP 414 is "Explicit Unicode Literal for Python 3.3" so I've no idea where the part about "tokenizing hook" comes from.  msg159865 refers to tabs in Django source and gives an example, then talks about "your GitHub repo where you published the hook".  Would someone care to enlighten me.
msg222713 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2014-07-10 23:25
> Would someone care to enlighten me.

It's supposed to be an install-time hook as mentioned in PEP 414 - see footnote no. 5 in the PEP.
msg222742 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-11 13:42
What are the Python core developers meant to do with this issue?  How does a piece of code on github relate to the core Python code in Mercurial?
msg222798 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2014-07-11 21:25
> How does a piece of code on github relate to the core Python code in Mercurial?

Tangentially. Armin Ronacher is the developer of both the GitHub code and the PEP 414 implementation, and it's referenced in the PEP. It doesn't make sense for any other Python core developer to worry about it. If Armin wants to close the issue, it's up to him.
msg222799 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2014-07-11 21:29
I hereby close this issue which is two years old.  The only point of the tokenizer thing was to support Python 3.2 which many libraries already have stopped supporting anyways.
History
Date User Action Args
2014-07-11 21:29:25aronachersetstatus: open -> closed
resolution: out of date
2014-07-11 21:29:16aronachersetmessages: + msg222799
2014-07-11 21:25:45vinay.sajipsetmessages: + msg222798
2014-07-11 13:42:50BreamoreBoysetmessages: + msg222742
2014-07-10 23:25:26vinay.sajipsetmessages: + msg222713
2014-07-10 18:23:39BreamoreBoysetnosy: + BreamoreBoy
messages: + msg222684
2012-06-03 20:54:40vinay.sajipsettitle: PEp 414 tokenizing hook does not preserve tabs -> PEP 414 tokenizing hook does not preserve tabs
2012-05-03 15:43:50vinay.sajipcreate