One observation is that when the -W option is used, the "warnings" module gets loaded very early and itself imports linecache which then imports tokenize. At this point, the standard IO streams have not been initialized (_io is imported later) and the builtin "open" may not exist at all. So when tokenize references "open", it is a NameError at that point (which explains why the "weird fix" above is actually correct).

As an additional annoyance, when importing Lib/warnings.py fails (for example if you sneak in a "1/0" at the beginning of the file), the exception gets silenced: the NameError above doesn't get printed.