classification
Title: ./configure --enable-optimizations && make && make install compiles Python twice
Type: Stage: resolved
Components: Build Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: --enable-optimizations makes common build commands always need to compile from scratch
View: 29243
Assigned To: Nosy List: haypo
Priority: normal Keywords:

Created on 2017-02-24 12:46 by haypo, last changed 2017-05-02 13:34 by haypo. This issue is now closed.

Messages (6)
msg288523 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-24 12:46
Hi,

I compiled Python 3.7 using PGO+LTO with "make && make install", but it seems like it lacks the _sysconfigdata module.


Moreover, I don't see any "Makefile" file in the installed directory, so the "~/installed/3.7/bin/python3  -S -m sysconfig --generate-posix-vars" command fails on trying to import _sysconfigdata_m_linux_x86_64-linux-gnu ...                  

$ find /home/haypo/installed/3.7 -name "Makefile"
<nothing>

Is it correct to use "make && make install" using PGO?

cd ~/cpython
make distclean ||:
./configure --enable-optimizations --with-lto --prefix=/home/haypo/installed/3.7
make clean
make
make install

haypo@speed-python$ ~/installed/3.7/bin/python3 
Failed to import the site module
Traceback (most recent call last):
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 544, in <module>
    main()
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 530, in main
    known_paths = addusersitepackages(known_paths)
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 282, in addusersitepackages
    user_site = getusersitepackages()
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 258, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 248, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "/home/haypo/installed/3.7/lib/python3.7/sysconfig.py", line 601, in get_config_var
    return get_config_vars().get(name)
  File "/home/haypo/installed/3.7/lib/python3.7/sysconfig.py", line 550, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/home/haypo/installed/3.7/lib/python3.7/sysconfig.py", line 421, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata_m_linux_x86_64-linux-gnu'
msg288524 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-24 12:58
PGO compilation uses "make profile-removal", maybe my issue is linked to that?

I'm sure that PGO compilation + "make install" worked two months, but using: make profile-opt && make install.
msg288525 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-24 13:03
The --enable-optimizations option of ./configure was added by the issue #26359 (first named --with-optimizations, it was recalled to --enable-optimizations later).

See also:
* issue #28032: "never imply --with-lto as part of --with-optimizations"
* issue #28605: Fix the help and What's New entry for --with-optimizations
msg288527 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-24 15:33
Ok, I confirm that it's possible to compile and install Python using PGO+LTO without --enable-optimizatioins:

cd ~/cpython
make distclean ||:
./configure --with-lto --prefix=/home/haypo/installed/3.7
make clean
make profile-opt
make install
msg292753 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-05-02 13:26
Ok, I tested again: in fact, "it works" but "make install" recompiles Python from scratch (1st build with -fprofile-generate, profile task, 2nd build with -fprofile-use) whereas "make" just succeeded to build Python!?
msg292754 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-05-02 13:34
Oh, it's just a duplicate of issue #29243.
History
Date User Action Args
2017-05-02 13:34:12hayposetstatus: open -> closed
superseder: --enable-optimizations makes common build commands always need to compile from scratch
messages: + msg292754

resolution: duplicate
stage: resolved
2017-05-02 13:26:59hayposetmessages: + msg292753
title: make install failure when using ./configure --enable-optimizations -> ./configure --enable-optimizations && make && make install compiles Python twice
2017-02-24 15:34:27hayposettitle: Missing _sysconfig_data when building+installating Python using PGO+LTO -> make install failure when using ./configure --enable-optimizations
2017-02-24 15:33:31hayposetmessages: + msg288527
2017-02-24 13:03:05hayposetmessages: + msg288525
2017-02-24 12:58:32hayposetmessages: + msg288524
2017-02-24 12:46:45haypocreate