classification
Title: lib2to3 pickles created with wrong permissions
Type: Stage:
Components: 2to3 (2.x to 3.x conversion tool) Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, loewis, ned.deily, r.david.murray, tpievila
Priority: normal Keywords:

Created on 2012-09-09 18:27 by tpievila, last changed 2012-09-10 06:23 by tpievila.

Messages (7)
msg170124 - (view) Author: Tomi Pieviläinen (tpievila) Date: 2012-09-09 18:27
The fix of #15822 caused another break. After installing 3.3rc2 from source, ls -ltc --full-time /usr/local/lib/python3.3/lib2to3/ shows:

-rw-------. 1 root root  1455 2012-09-09 20:55:36.475519602 +0300 PatternGrammar3.3.0.candidate.2.pickle
-rw-------. 1 root root 20862 2012-09-09 20:55:36.425520211 +0300 Grammar3.3.0.candidate.2.pickle
drwx------. 2 root root  4096 2012-09-09 20:55:33.681520171 +0300 __pycache__/
drwxr-xr-x. 4 root root  4096 2012-09-09 20:55:30.087520195 +0300 tests/
drwxr-xr-x. 3 root root  4096 2012-09-09 20:55:30.063457702 +0300 pgen2/
drwxr-xr-x. 3 root root  4096 2012-09-09 20:55:30.028520614 +0300 fixes/
-rw-r--r--. 1 root root 28046 2012-09-09 20:55:09.319519773 +0300 refactor.py
-rw-r--r--. 1 root root 28386 2012-09-09 20:55:09.279456945 +0300 pytree.py
-rw-r--r--. 1 root root  1114 2012-09-09 20:55:09.242458183 +0300 pygram.py
-rw-r--r--. 1 root root   793 2012-09-09 20:55:09.205457004 +0300 PatternGrammar.txt
-rw-r--r--. 1 root root  7075 2012-09-09 20:55:09.168457078 +0300 patcomp.py
-rw-r--r--. 1 root root 11624 2012-09-09 20:55:09.131457019 +0300 main.py
-rw-r--r--. 1 root root    67 2012-09-09 20:55:09.091644598 +0300 __main__.py
-rw-r--r--. 1 root root     7 2012-09-09 20:55:09.052457036 +0300 __init__.py
-rw-r--r--. 1 root root  6589 2012-09-09 20:55:09.014519602 +0300 Grammar.txt
-rw-r--r--. 1 root root 14533 2012-09-09 20:55:08.977519642 +0300 fixer_util.py
-rw-r--r--. 1 root root  6706 2012-09-09 20:55:08.940519615 +0300 fixer_base.py
-rw-r--r--. 1 root root 10011 2012-09-09 20:55:08.903519608 +0300 btm_utils.py
-rw-r--r--. 1 root root  6833 2012-09-09 20:55:08.865457147 +0300 btm_matcher.py


The pickles are now correctly newer than the Grammar texts, but are readable only by root. Quick chmod fixes this, but has to be done manually on systems with strict UMASK for root.
msg170125 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-09-09 18:44
I propose to postpone the fix for that to 3.3.1. Meanwhile, users should either set umask to 022 before installing, or chmod the files afterwards.
msg170127 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-09-09 19:19
AFAIK, the pickles are created with exactly the same permissions as the cached pyc and pyo files have been all along.  So, if you propose to change the permissions for the pickles, the compileall steps should be changed as well.
msg170130 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-09-09 19:46
BTW, AFAICT, you'll get exactly the same behavior (with distribute failing due to pickle write permission) with earlier releases, like 3.2.3, if you happen to do a "make install" with an inappropriate umask value.  So it appears that setting umask properly has been an existing, if perhaps undocumented, requirement.  Ensuring permissions for all installed files are set to sane values seems like a good idea, though.
msg170131 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-09-09 20:06
It occurs to me that ignoring the user's selected umask could be considered a bug by some people.  A package manager for a distribution should presumably ensure the correct umask during install, but if a user is doing a direct install, it seems reasonable to think they are responsible for setting the umask they want used.  There is a fuzzy line here, though, because distutils is also a bit of a package manager...

Regardless, though, Ned's observation seems to indicate this is not a regression.  Tomi, do you find a different behavior in 3.2?
msg170143 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-09-09 23:04
It appears that it is tradition of (BSD) install(1) to ignore umask. For consistency, all installed files should get the same ACL (+-x where necessary). So I think this behavior is a bug (although apparently not a regression).
msg170159 - (view) Author: Tomi Pieviläinen (tpievila) Date: 2012-09-10 06:23
> Tomi, do you find a different behavior in 3.2?

No, the behaviour is the same in 2.6-3.2: no compiled files have permissions for other than root, and all non-compiled files do (because install is run with "-m 644"). What made me report this bug was that 3.3rc1 does not have this behaviour: while the pickles had a bad timestamp, the permissions were just fine, so I assumed that someone had changed the this intentionally.

Also this hasn't been a problem in Python2.x. I can install 2.6.8 and 2.7.3 from source without fixing any permissions, and still have a working Python installation. 3.2.3 requires chmoding the lib2to3 pickles, but no other compiled files need to be touched (it seems that they aren't really needed).
History
Date User Action Args
2012-09-10 06:23:01tpievilasetmessages: + msg170159
2012-09-09 23:04:02loewissetmessages: + msg170143
2012-09-09 20:06:07r.david.murraysetnosy: + r.david.murray
messages: + msg170131
2012-09-09 19:46:21ned.deilysetmessages: + msg170130
2012-09-09 19:19:33ned.deilysetmessages: + msg170127
2012-09-09 19:15:43Arfreversetnosy: + ned.deily, Arfrever

versions: + Python 2.7, Python 3.2
2012-09-09 18:44:24loewissetnosy: + loewis
messages: + msg170125
2012-09-09 18:27:06tpievilacreate