On Sun, Nov 7, 2010 at 8:47 PM, STINNER Victor <report@bugs.python.org> wrote:
>
> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
>
> +            try:
> +                with open(filename, 'rb') as fp:
> +                    encoding, _ = tokenize.detect_encoding(fp.readline)
> +            except IOError:
> +                encoding = None
>
> You should use 'utf-8' instead of None (which will fall back to the locale encoding) here.
>

I know.  I was too lazy to look up the correct spelling for the proof
of concept.  I am really posting this patch to show how this bug can
be fixed in theory and discuss how much of refactoring is acceptable.
For example, do we need to preserve trace.find_strings() function?
What is the best way to pass around source code? - file-like objects,
line iterators, readline-like function?  Also compile(prog, filename,
"exec") to find module's bytecode is a hack.  There must be a standard
way to achieve that which would use a .pyc file if available.