This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: during 'make install', pre-existing site-packages residents are recompiled. Twice.
Type: Stage:
Components: Build, Extension Modules, Installation Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: MattDMo, r.david.murray
Priority: normal Keywords:

Created on 2016-08-28 14:53 by MattDMo, last changed 2022-04-11 14:58 by admin.

Messages (2)
msg273824 - (view) Author: Matt Morrison (MattDMo) * Date: 2016-08-28 14:53
I've been building Python 3 from source on Linux for a long time (at least since 3.2 days), and I honestly can't remember if this has happened before 3.6 or not, which is why I'm only tagging this 3.6. 

Basically, when running 'make install' (after building either static or shared, I haven't been messing around with any other options except install prefix - $HOME) you'll start seeing messages like:

=====
Listing '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/__init__.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/anova_nistcertified.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/ar_panel.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/example_kernridge.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/gmm.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/kernridgeregress_class.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/ols_anova_original.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/onewaygls.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/penalized.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/predstd.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/runmnl.py'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/sympy_diff.py'...
Listing '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/tests'...
Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/tests/__init__.py'...
=====

and on and on and on. I usually don't pay much attention to them, but since I have nothing better to do I just stare idly at them and watch as all my third-party modules in 'site-packages' scroll by in alphabetical order very quickly. For the last few builds of 3.6, though, I've noticed that the modules get gone through twice. I have a pretty large number of modules (341, according to `ls | wc -l`), so it takes a while to go through them, and they're definitely being repeated.

Unfortunately, I don't know a whole lot about Makefiles or the internals of the installation process, so I don't have the slightest idea of where to begin even looking for the bug. But, I bet you ladies and gentlemen do :)

If it's necessary, I can rebuild and pipe all the output into a file for you to examine. Just let me know.
msg273825 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-08-28 15:09
As far as I can tell, this is working as designed, and in fact you ought to see site-packages getting compiled *three* times (normal, -O, and -OO).

I wonder if this is a holdover from when we didn't have per-interpreter pyc files, and its just that no one noticed the need for recompile of site-packages didn't exist any more.  On the other hand, I may be missing some piece of logic that means that it is still needed.
History
Date User Action Args
2022-04-11 14:58:35adminsetgithub: 72071
2016-08-28 15:09:22r.david.murraysetnosy: + r.david.murray
messages: + msg273825
2016-08-28 14:53:14MattDMocreate