It is not when freezing the logging package, so any use of the logging package fails:

>>> import logging
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "<pyrun>/importlib/_bootstrap.py", line 2237, in _find_and_load
  File "<pyrun>/importlib/_bootstrap.py", line 2226, in _find_and_load_unlocked
  File "<pyrun>/importlib/_bootstrap.py", line 1200, in _load_unlocked
  File "<pyrun>/importlib/_bootstrap.py", line 1129, in _exec
  File "<pyrun>/importlib/_bootstrap.py", line 1359, in exec_module
  File "<pyrun>/logging/__init__.py", line 61, in <module>
NameError: name '__file__' is not defined

This is the code in question:

#
# _srcfile is used when walking the stack to check when we've got the first
# caller stack frame.
#
if hasattr(sys, 'frozen'): #support for py2exe
    _srcfile = "logging%s__init__%s" % (os.sep, __file__[-4:])
else:
    _srcfile = __file__
_srcfile = os.path.normcase(_srcfile)

Note the special case for py2exe. But this doesn't really help, since frozen modules don't have the __file__ attribute set - at least not when generated with Tools/freeze.

PS: This is with Python 3.4.1.

The issue is similar to Issue20884.

New changeset 11a920a26f13 by Vinay Sajip in branch '3.4':
Issue #21709: Remove references to __file__ when part of a frozen application.
http://hg.python.org/cpython/rev/11a920a26f13

New changeset 149cc6364180 by Vinay Sajip in branch 'default':
Closes #21709: Merged fix from 3.4.
http://hg.python.org/cpython/rev/149cc6364180

Hi Vinaj,

thanks for the patch, but it doesn't really help outside of py2exe. The sys.frozen flag is not an official Python API and it's unlikely to become one, since you can freeze the whole application or just parts of it, which sys.frozen would not be able to address.

Instead, the modules in the stdlib should be aware of the fact that __file__ is not always available and provide fallback solutions.

Could you please use a fix that works for Python tools in general ?

E.g. instead of doing an equal test inf .findCaller() it may be better to use a regular expression or you could patch the __init__ module's co_filename into the module as _srcfile (after it's fully initialized).

Thanks.

> Could you please use a fix that works for Python tools in general?

I suggested an alternative implementation altogether in Issue #16778, but it was suggested that we wait for frame annotations. I'm not sure what the schedule for that is.

> The sys.frozen flag is not an official Python API and it's unlikely to become one

Would using imp.is_frozen('logging') rather than hasattr(sys, 'frozen') meet your requirement here? I'm not saying it's the ideal solution, but perhaps it will do until frame annotations arrive and we can avoid using filenames altogether?

On 11.06.2014 11:25, Vinay Sajip wrote:
> 
> Vinay Sajip added the comment:
> 
>> Could you please use a fix that works for Python tools in general?
> 
> I suggested an alternative implementation altogether in Issue #16778, but it was suggested that we wait for frame annotations. I'm not sure what the schedule for that is.
> 
>> The sys.frozen flag is not an official Python API and it's unlikely to become one
> 
> Would using imp.is_frozen('logging') rather than hasattr(sys, 'frozen') meet your requirement here? I'm not saying it's the ideal solution, but perhaps it will do until frame annotations arrive and we can avoid using filenames altogether?

I don't think any of this is needed here. _srcfile is only used to
identify the caller's stack frame and needs to be set to the co_filename
of the stack frame used by the logging.__init__ module.

Here's a sketch of what I had hinted at in my last reply:

def _get_module_filename():
    return getLogger.func_code.co_filename

You simply use the .co_filename attribute of one of the module's functions
to get useable value for __file__.

> _srcfile is only used to identify the caller's stack frame

Not quite. It's also used to indicate whether findCaller() should be called at all: setting it to None avoids calling findCaller(), which might be desirable in some performance-sensitive scenarios.

So if you mean "just call _get_module_filename() instead of accessing _srcFile", that won't do. If you mean "set _srcFile to the return value of _get_module_filename()", that might work, if I e.g. move the _srcFile definition to after addLevelName (say) and do just

_srcFile = addLevelName.__code__.co_filename

How does that sound?

On 11.06.2014 12:32, Vinay Sajip wrote:
> 
> Vinay Sajip added the comment:
> 
>> _srcfile is only used to identify the caller's stack frame
> 
> Not quite. It's also used to indicate whether findCaller() should be called at all: setting it to None avoids calling findCaller(), which might be desirable in some performance-sensitive scenarios.
> 
> So if you mean "just call _get_module_filename() instead of accessing _srcFile", that won't do. If you mean "set _srcFile to the return value of _get_module_filename()", that might work, if I e.g. move the _srcFile definition to after addLevelName (say) and do just
> 
> _srcFile = addLevelName.__code__.co_filename
> 
> How does that sound?

That's what I meant, yes. Please also add some comment explaining
why this is done in this way.

FWIW: Given that __file__ is not always set, it may be worthwhile
introducing some generic helper to the stdlib which uses the
.co_filename attribute to get the compile time filename as fallback
in case __file__ is not set.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

> Please also add some comment explaining why this is done in this way.

Natürlich :-)

> it may be worthwhile introducing some generic helper to the stdlib

Wouldn't you have to pass in a function (or code object) from a specific module, though? It seems more logical to have __file__ set, even for frozen modules (after all, if it's there in a code object's co_filename, is there some reason it shouldn't be exposed as a module attribute? (Even though it isn't at the moment.)

On 11.06.2014 13:22, Vinay Sajip wrote:
> 
> Vinay Sajip added the comment:
> 
>> Please also add some comment explaining why this is done in this way.
> 
> Natürlich :-)

Prima :-)

>> it may be worthwhile introducing some generic helper to the stdlib
> 
> Wouldn't you have to pass in a function (or code object) from a specific module, though? It seems more logical to have __file__ set, even for frozen modules (after all, if it's there in a code object's co_filename, is there some reason it shouldn't be exposed as a module attribute? (Even though it isn't at the moment.)

Well, I guess passing in a reference to the module would suffice. The
function could then look around for functions, methods, etc. to find
a usable code object.

I agree that having a __file__ attribute in frozen modules would
be helpful, since it's obviously not widely known that this
attribute does not always exist (just grep the stdlib as example,
in particular the test suite).

The module object's PyModule_GetFilenameObject() even reports a missing
__file__ attribute as a SystemError.

Perhaps something to discuss on python-dev.

While the current patch does not resolve the issue, I'm leaving the issue closed and have instead opened a new Issue21736 which tracks the idea to add a __file__ attribute to frozen modules per default.

> While the current patch does not resolve the issue, I'm leaving the issue closed

That's fine - I will implement the changes we discussed in this issue, even if it's something of a stop-gap.

New changeset bec6f18dd636 by Vinay Sajip in branch '3.4':
Issue #21709: Improved implementation to cover the frozen module case.
http://hg.python.org/cpython/rev/bec6f18dd636

New changeset bd44ad77013a by Vinay Sajip in branch 'default':
Issue #21709: Merged update from 3.4.
http://hg.python.org/cpython/rev/bd44ad77013a

> addLevelName.__code__.co_filename

Isn't __code__ implementation-specific?

> Isn't __code__ implementation-specific?

It is, but ISTM it's worth getting a resolution for #21736 before another patch for this issue is developed.

> Isn't __code__ implementation-specific?

Further data point: the contents of __code__ might be implementation-specific (as there are no other Python 3 implementations), but the 2.x equivalent, func_code, is to be found in CPython, Jython, IronPython and PyPy, and in each case it has a co_filename attribute pointing to a source file. Other internal details no doubt differ.

New changeset da971f467f6e by Gregory P. Smith in branch '2.7':
Issue #21709: Fix the logging module to not depend upon __file__ being set
https://hg.python.org/cpython/rev/da971f467f6e

New changeset bdb219a0219c by Gregory P. Smith in branch '2.7':
Issue21709: Call os.path.normcase when setting _srcfile for compatibility
https://hg.python.org/cpython/rev/bdb219a0219c

I applied the same _srcfile setting cleanup to 2.7 as we ran into this at work with an embedded python stdlib.  __file__ was set to something by the embedded importer, but to something different than the path seen on the actual code objects which appear to be based on the path seen during pyc compilation done at build time.