In debugging an intermittent test failure we discovered that if '.' is
in the path and import_module is called, then what happens when
__import__ causes the creation of a .pyc file changes.  Without the
forgoing, the permissions of the .py file are copied to the .pyc file
(although the fix for issue 6070 removes the execute bits if set).  With
the forgoing, the write bit is set on the .pyc file even if it was not
set on the .py file.

The second behavior is how things always worked before 2.6. (Why it
changed in 2.6 I don't know, and I'm not convinced it was a change for
the better; see also issue 6074).

I will upload a test as soon as I create the issue and get an issue
number to label it with.

Note that this issue exists only 3.x, even though trunk uses the
backported import_module function from importlib.

Test case.

import_module are implemented completely differently between 2.6 and 3.1, 
so the difference is not surprising.

As for the permission stuff, the files are simply created using open(..., 
'wb') so it has everything to do with what the system default for the 
executing user happens to be. What exactly are the permissions you are 
trying to set all bytecode files to?

If you have a .py file with permissions r--r--r-- and you import it, the
pyc file will (in 2.6+ and 3.x) have permissions r--r--r--.  However, if
'.' is in the path and import_module has been used, the file will
instead have permissions rw--r--r-- (assuming the umask is 022).

Does that make the issue clearer?

As indicated, this is a low priority bug :)

Oh, a bit of clarification: the call that creates the pyc file in both
the "normal" case and the error case is a call to the normal import
command (or __import__ in the test case). The call to import_module is a
prereq to producing the bug, but it doesn't matter what module it
imports (as long as it hasn't been previously imported).  The import
that shows the behavior imports a TESTFN module in the test case.

You might want to load up the test case and play with it.  I'm
completely mystified as to how import_module could be affecting the
regular import semantics...I'm guessing it is a subtle side effect of
something unexpected ;)

On Mon, Jul 20, 2009 at 16:25, R. David Murray <report@bugs.python.org>wrote:

>
> R. David Murray <rdmurray@bitdance.com> added the comment:
>
> Oh, a bit of clarification: the call that creates the pyc file in both
> the "normal" case and the error case is a call to the normal import
> command (or __import__ in the test case). The call to import_module is a
> prereq to producing the bug, but it doesn't matter what module it
> imports (as long as it hasn't been previously imported).  The import
> that shows the behavior imports a TESTFN module in the test case.
>
> You might want to load up the test case and play with it.  I'm
> completely mystified as to how import_module could be affecting the
> regular import semantics...I'm guessing it is a subtle side effect of
> something unexpected ;)

It's beyond mystifying as both importlib.__import__ and
importlib.import_module are thin wrappers that do nothing but splice strings
for the same function that does the heavy lifting. But I will see if I ever
find time to work on this.

So removing the built-in, frozen, and extension importers did not stop the 
bug from happening. Calling importlib._bootstrap._PyFileFinder directly 
does not trigger the bug, even when trying with a finder for '.' first. 
And having sys.path be only '.' for fileinput still triggers the bug.

On another day, the next step is to start stripping out stuff in importlib 
to see what the minimal thing is that triggers the bug.

OK, I need a double-check here, David. At this point I have narrowed it 
down to this code triggering it::

  finder = importlib._bootstrap._PyPycFileFinder('.')
  sys.path_importer_cache['.'] = finder

And I am not kidding, that assignment is required. I might be able to 
narrow the code down further in _PyPycFileFinder, but I wanted to double-
check that I have not gone insane and that these two lines do indeed 
trigger the problem.

Turns out the failure is because I have simply been opening bytecode 
files for writing using _io.FileIO(..., 'w') which just uses the OS's 
default permissions. Guess that won't cut it anymore. =)

So does this mean I am expected to chmod the bytecode file to have 
matching read and write bits to the source but no execution bits 
regardless of the source file? And what about writing new bytecode 
files? I am not about to ignore permissions and simply write over files 
just because I can as someone could have set them independently of 
Python to be read-only.

Let me know that I am doing the right thing, David.

Well, the problem with asking me what the right thing to do is that I
think the old behavior of just using the default os umask makes the most
sense.  I think we should figure out who made the original change in
behavior and ask them what the right thing is to do and why.  I'll see
if I can dig up the 'who' tomorrow.

Did this go anywhere, David? Since beta2 just went out now is the time to either revert or enshrine the new behavior.

At some point this was fixed in importlib.