This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author gdr@garethrees.org
Recipients ezio.melotti, gdr@garethrees.org, sandro.tosi
Date 2011-08-04.23:01:00
SpamBayes Score 2.7577107e-11
Marked as misclassified No
Message-id <1312498861.23.0.594872824679.issue12691@psf.upfronthosting.co.za>
In-reply-to
Content
See my last paragraph: I propose to deliver a single patch that fixes both this bug and issue12675. I hope this is OK. (If you prefer, I'll try to split the patch in two.)

I just noticed another bug in untokenize(): in compatibility mode, if untokenize() is passed an iterator rather than a list, then the first token gets discarded:

    Python 3.3.0a0 (default:c099ba0a278e, Aug  2 2011, 12:35:03) 
    [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from tokenize import untokenize
    >>> from token import *
    >>> untokenize([(NAME, 'hello')])
    'hello '
    >>> untokenize(iter([(NAME, 'hello')]))
    ''

No-one's noticed this because the unit tests only ever pass lists to untokenize().
History
Date User Action Args
2011-08-04 23:01:01gdr@garethrees.orgsetrecipients: + gdr@garethrees.org, ezio.melotti, sandro.tosi
2011-08-04 23:01:01gdr@garethrees.orgsetmessageid: <1312498861.23.0.594872824679.issue12691@psf.upfronthosting.co.za>
2011-08-04 23:01:00gdr@garethrees.orglinkissue12691 messages
2011-08-04 23:01:00gdr@garethrees.orgcreate