➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author r.david.murray
Recipients Tyler.Crompton, r.david.murray
Date 2013-02-04.17:06:59
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1359997619.95.0.524049088253.issue17125@psf.upfronthosting.co.za>
In-reply-to
Content 
The docs could certainly be more explicit...currently they state that tokenize is *detecting* the encoding of the file, which *implies* but does not make explicit that the input must be binary, not text.

The doc problem will get fixed as part of the fix to issue 12486, so I'm closing this as a duplicate.  If you want to help out with a patch review and doc patch suggestions on that issue, that would be great.
History
Date User Action Args
2013-02-04 17:06:59r.david.murraysetrecipients: + r.david.murray, Tyler.Crompton
2013-02-04 17:06:59r.david.murraysetmessageid: <1359997619.95.0.524049088253.issue17125@psf.upfronthosting.co.za>
2013-02-04 17:06:59r.david.murraylinkissue17125 messages
2013-02-04 17:06:59r.david.murraycreate