__file__ is the filename from which the module *was* loaded (the inspect doc [1] should probably reflect that [2]).  The import machinery only uses the module's __spec__ (origin, etc.).  __file__ is set strictly as informational (and for backward-compatibility).

Per the language reference [3], __file__ may be omitted when it does not have semantic meaning.  It also says "Ultimately, the loader is what makes use of __file__", but that hasn't been accurate since PEP 451 landed. [4]  Notably, though, for now the module __repr__() *does* use __file__ if it is available (and the loader doesn't implement module_repr).  The counterpart of __file__ within a module's spec is __spec__.origin.  The two should stay in sync.  In the case of frozen modules origin is set to "frozen".

Giving __file__ to frozen modules is inaccurate.  The file probably won't be there and the module certainly wasn't loaded from that location.

Stdlib modules should not rely on all module's having __file__.  Removing __file__ from frozen modules was a change in 3.4 and I'd consider it a 3.4 bug if any stdlib module relied on it.  For that matter, I'd consider it a bug if a module relied on all modules having __file__ or even __file__ being set to an actual filename.

Would it be inappropriate to set an additional informational attribute on frozen modules to indicate the original path?  Something like __frozen_filename__.  Then you wouldn't need to rely on __code__.co_filename.

p.s. Searching for __file__ in the docs [5] illustrates its prevalence.

[1] https://docs.python.org/3/library/inspect.html#types-and-members
[2] issue #21760
[3] https://docs.python.org/3/reference/import.html#__file__
[4] issue #21761
[5] https://docs.python.org/3/search.html?q=__file__