Author r.david.murray
Recipients Tyler.Crompton, r.david.murray
Date 2013-02-04.17:06:59
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
The docs could certainly be more explicit...currently they state that tokenize is *detecting* the encoding of the file, which *implies* but does not make explicit that the input must be binary, not text.

The doc problem will get fixed as part of the fix to issue 12486, so I'm closing this as a duplicate.  If you want to help out with a patch review and doc patch suggestions on that issue, that would be great.
Date User Action Args
2013-02-04 17:06:59r.david.murraysetrecipients: + r.david.murray, Tyler.Crompton
2013-02-04 17:06:59r.david.murraysetmessageid: <>
2013-02-04 17:06:59r.david.murraylinkissue17125 messages
2013-02-04 17:06:59r.david.murraycreate