classification
Title: Python 2.6 makes .pyc/.pyo bytecode files executable
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: r.david.murray Nosy List: brett.cannon, georg.brandl, joe.amenta, loewis, markon, phd, r.david.murray, tim.golden
Priority: low Keywords: patch

Created on 2009-05-20 10:58 by phd, last changed 2009-07-19 02:29 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
import_patch2.patch markon, 2009-05-26 07:28
issue6070.patch r.david.murray, 2009-06-22 22:29
Messages (26)
msg88112 - (view) Author: Oleg Broytman (phd) * Date: 2009-05-20 10:58
On compilation of .pyc/.pyo bytecode files on import Python 2.6 copies
Unix file access attributes (-rwx-) from the imported file. I'd think
it's ok except for executable (-x-) bit - bytecode files are not
directly executable. That is, for a module.py with attributes -rwxr-x---
I expect python2.6 -c 'import module' would produce module.pyc with
attributes -rw-r-----.

python compileall.py . saves compiled files without the executable bit;
it doesn't copy attributes - it just saves files and saving obeys umask.
python2.5 -c 'import module.py' doesn't copy the executable bit, it
obeys umask too. I consider this as a regression in 2.6. Please mask out
executable bits on bytecode files saving.
msg88208 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-05-22 19:15
Would you like to provide a patch?
msg88211 - (view) Author: Joe Amenta (joe.amenta) Date: 2009-05-22 19:54
Python writes compiled files with the same file permissions as the
source module.  In your specific example:

$ python2.6 -c 'import module'

This will produce module.pyc with the same attributes as module.py

While I am not familiar with modifying C stat structs, I can say that in
trunk/Python/import.c, somewhere after line 977 but before line 1014,
the stat struct "st" must be examined for S_IXUSR, S_IXGRP, and S_IXOTH
and have those bits removed in order to never produce a compiled module
with "x" in the stat.  I would assign low priority to this issue, as it
has little impact on the behavior of Python, even though it might cause
unintended consequences to a user trying to execute the bytecode, if the
user interprets the "x" to mean that it is executable by the system.
msg88216 - (view) Author: Oleg Broytman (phd) * Date: 2009-05-22 20:32
I will try to look at import.c, though I must say I am a bad C
programmer. I have switched to Python after ten years of Pascal.

Low priority is ok.
msg88260 - (view) Author: Marco Buccini (markon) Date: 2009-05-24 14:10
I attach a patch to correct this little bug.

Bye ;)
msg88279 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-05-24 20:09
Are these S_IX... constants available on every platform we support?
msg88281 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-05-24 20:39
No, they are not supported on Visual Studio.
msg88321 - (view) Author: Marco Buccini (markon) Date: 2009-05-25 19:01
TO georg.brandl:
I remembered that Windows wasn't POSIX compliant, but...I thought they
used the same sys/stat.h constants.
I was wrong :P

TO loewis:
ok, I've added a new patch.
Since I've never written any code for Windows, can you check if it works
fine now?
I've added a simple #ifdef WINDOWS,...
msg88325 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-05-25 21:43
The patch of file 14070 doesn't compile, but I get the idea. I won't
have time to test it in the next few days or weeks, though.
msg88337 - (view) Author: Marco Buccini (markon) Date: 2009-05-26 07:28
TO loewis: 
this new patch works fine.
I've removed the first #endif.

thank you ;)
msg89602 - (view) Author: Oleg Broytman (phd) * Date: 2009-06-22 15:21
import_patch2.patch doesn't work for me. I patched and compiled Python
2.6.2 and without installing it ran ./python -c "import test" in the
build directory. It copied executable bits from test.py to test.pyc.
msg89603 - (view) Author: Marco Buccini (markon) Date: 2009-06-22 17:19
hmm.. the problem is that Windows doesn't support well permissions as
all the other POSIX compliant OSs ...
I've searched for a solution on the web, and I've found a complete
answer on:
http://stackoverflow.com/questions/592448/c-how-to-set-file-permissions-cross-platform

The patch doesn't work well since it only checks for User's permissions
so it works well for that. 
Maybe using the Windows API you can change the permissions as you want.
But since I don't know them, I can't help anymore :(
msg89604 - (view) Author: Oleg Broytman (phd) * Date: 2009-06-22 17:25
I am not on Windows. I am on Linux.
msg89615 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-06-22 22:29
The patch did not apply for me.  I modified the code by hand based on
the patch file, and on Gentoo linux it worked for me.  Patch that
applies cleanly to trunk attached.
msg89624 - (view) Author: Marco Buccini (markon) Date: 2009-06-23 07:59
Thank you David.. sorry for my errors :)
but this is my first patch :P 

I've recompiled py2.6 and it works fine on my Debian.

As you can see:
-rwxr-xr-x  1 marco marco    81822 30 set  2008 setup.py

./python -c "import setup"
-rw-r--r--  1 marco marco    42156 23 giu 09:58 setup.pyc


However, the version 2.5 doesn't have this "bug" and I've not recompiled it.
msg89727 - (view) Author: Oleg Broytman (phd) * Date: 2009-06-26 11:59
Sorry, found the bug in my process of testing. ./python uses /usr/lib/python26.so instead of ./python26.so. Setting LD_LIBRARY_PATH=. fixes the problem and the test passes. The patch is ok.
msg89729 - (view) Author: Marco Buccini (markon) Date: 2009-06-26 14:21
Thank you for your report :P

Otherwise I really didn't know how solve it, thank you :P

However, on *NIX-like systems it can work well; 
but can someone try it on Windows? Since I know that only NTFS (and
versions >= XP) supports permissions, if a user have a FATfs (or Win98,
95,..) I think it raises error.
msg89730 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2009-06-26 15:27
Making something executable on Windows has nothing to
do with file permissions. You can set them as much
as you like, but executability is determined by file
associations, possibly in association with PATHEXT
settings. AFAICT, the current Python installer does
this already for pyc / pyo files. (When I say "nothing
to do with file permissions", obviously the file in
question must be readable)
msg89731 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-06-26 15:36
So we should just do #ifndef MS_WINDOWS?  Do we have any non-windows
non-posix platforms?  (People on #python-dev don't think so.)
msg89732 - (view) Author: Oleg Broytman (phd) * Date: 2009-06-26 16:17
I can confirm the patch works on WinXP on NTFS partition and Samba-shared network drive. I have WinXP running in an emulator (VirtualBox) and I compiled Python using MSVC90 Express.
msg89733 - (view) Author: Oleg Broytman (phd) * Date: 2009-06-26 16:17
Yes, I think #ifndef MS_WINDOWS is enough.
msg90208 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-07-07 01:10
#ifndef version of the patch applied to trunk in r73870, with tests. 
Leaving open until I merge it.
msg90221 - (view) Author: Marco Buccini (markon) Date: 2009-07-07 09:28
@r.david.murray:
Does this works on Windows? Are you sure Oleg? :)
Since you've done this:
#ifndef MS_WINDOWS
 /* mode = ..*/
#endif

but on Windows the compiler "jumps" over this code, so you can get a
binding error, since it doesn't find the variable "mode"...

E.g (on my Debian):

#ifndef __GNUC__
 #define X 10
#endif

int main()
{
    printf("%d\n", X);
}

it gives error: ‘X’ undeclared (first use in this function)
msg90222 - (view) Author: Oleg Broytman (phd) * Date: 2009-07-07 09:33
I didn't test ifndef-version, I tested the full version (issue6070.patch)
on both Linux and w32. You are right, 'mode' must be defined even on w32.
msg90223 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-07-07 09:49
Drat. I should have uploaded the revised patch first for testing.  I'll
update it to the tested patch.
msg90699 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-07-19 02:29
Committed to 2.6 in r74085, py3k in r74058 (by alexandre vassalotti),
and 3.1 in r74086.
History
Date User Action Args
2009-09-29 12:15:12r.david.murraylinkissue7016 superseder
2009-07-19 02:29:25r.david.murraysetstatus: open -> closed

messages: + msg90699
2009-07-07 09:49:20r.david.murraysetmessages: + msg90223
2009-07-07 09:33:26phdsetmessages: + msg90222
2009-07-07 09:28:23markonsetmessages: + msg90221
2009-07-07 01:10:21r.david.murraysetnosy: + brett.cannon
messages: + msg90208

assignee: r.david.murray
resolution: fixed
stage: test needed -> resolved
2009-06-26 16:17:54phdsetmessages: + msg89733
2009-06-26 16:17:25phdsetmessages: + msg89732
2009-06-26 15:36:58r.david.murraysetmessages: + msg89731
2009-06-26 15:27:10tim.goldensetnosy: + tim.golden
messages: + msg89730
2009-06-26 14:21:34markonsetmessages: + msg89729
2009-06-26 11:59:54phdsetmessages: + msg89727
2009-06-23 07:59:41markonsetmessages: + msg89624
2009-06-22 22:29:15r.david.murraysetfiles: + issue6070.patch
priority: low

versions: + Python 3.1, Python 2.7, Python 3.2
nosy: + r.david.murray

messages: + msg89615
stage: test needed
2009-06-22 17:25:41phdsetmessages: + msg89604
2009-06-22 17:19:08markonsetmessages: + msg89603
2009-06-22 15:21:26phdsetmessages: + msg89602
2009-05-26 12:32:22markonsetfiles: - import_patch2.patch
2009-05-26 12:32:18markonsetfiles: - import_patch.c
2009-05-26 07:28:25markonsetfiles: + import_patch2.patch

messages: + msg88337
2009-05-25 21:43:19loewissetmessages: + msg88325
2009-05-25 19:04:08markonsetfiles: + import_patch2.patch
2009-05-25 19:03:59markonsetfiles: - import_patch2.patch
2009-05-25 19:01:17markonsetfiles: + import_patch2.patch
keywords: + patch
messages: + msg88321
2009-05-24 20:39:59loewissetmessages: + msg88281
2009-05-24 20:09:04georg.brandlsetnosy: + georg.brandl
messages: + msg88279
2009-05-24 17:07:53loewissettitle: Pyhon 2.6 makes .pyc/.pyo bytecode files executable -> Python 2.6 makes .pyc/.pyo bytecode files executable
2009-05-24 14:10:24markonsetfiles: + import_patch.c
nosy: + markon
messages: + msg88260

2009-05-22 20:32:10phdsetmessages: + msg88216
2009-05-22 19:54:02joe.amentasetnosy: + joe.amenta
messages: + msg88211
2009-05-22 19:15:09loewissetnosy: + loewis
messages: + msg88208
2009-05-20 10:58:48phdcreate