It looks like the problem of fix_latin.diff is the decoding_state: 
it's set to STATE_NORMAL whereas current behaviour is to stay in state 
STATE_RAW.

I wrote another patch which is a mix of case 1 (utf-8: just set 
tok->encoding) and case 2 (another charset: set tok->enc, 
tok->encoding and tok>decoding_state): a new case 3 which set enc, 
encoding but stay a the state STATE_RAW. I don't understand my patch, 
so review it (twice or more :-D). Using my patch:
 - compile(...) works
 - test_shlex.py works
 - test_pep3120.py
 - iso.py works