I'm trying to build 3.7.0 locally (not as superuser) on a server. I am able to successfully build without `--enable-optimizations` enabled, but the build fails with it enabled.

I have dependencies installed in custom locations as well, and this is my environment.

PREF=/scratch

export LC_ALL=C
export CC=gcc
export CXX=g++
    
export PATH=/opt/gcc/4.9.2/bin:$PREF/binutils/bin:/bin:/usr/bin
    
export LD_LIBRARY_PATH=/opt/gcc/4.9.2/lib64:/opt/gcc/4.9.2/lib:$PREF/bzip2/lib:$PREF/gdbm/lib:$PREF/libffi/lib64:$PREF/ncurses/lib:$PREF/openmpi/lib:$PREF/openssl/lib:$PREF/readline/lib:$PREF/sqlite/lib:$PREF/util-linux/lib:$PREF/xz/lib:$PREF/zlib/lib:/usr/lib64
    
export CPATH=/opt/gcc/4.9.2/include:$PREF/bzip2/include:$PREF/gdbm/include:$PREF/libffi/include:$PREF/ncurse
s/include:$PREF/openmpi/include:$PREF/openssl/include:$PREF/readline/include:$PREF/sqlite/include:$PREF/util
-linux/include:$PREF/xz/include:$PREF/zlib/include

I also have to update setup.py to point to the right locations for dependencies. This is the diff, and I've attached the full file.

46c46
< disabled_module_list = ["_tkinter", "_uuid"]
---
> disabled_module_list = []
544,567c544,545
<             add_dir_to_list(self.compiler.library_dirs,
<                             '/scratch/readline/lib')
<             add_dir_to_list(self.compiler.include_dirs,
<                             '/scratch/readline/include')
<             add_dir_to_list(self.compiler.library_dirs,
<                             '/scratch/bzip2/lib')
<             add_dir_to_list(self.compiler.include_dirs,
<                             '/scratch/bzip2/include')
<             add_dir_to_list(self.compiler.library_dirs,
<                             '/scratch/ncurses/lib')
<             add_dir_to_list(self.compiler.include_dirs,
<                             '/scratch/ncurses/include')
<             add_dir_to_list(self.compiler.library_dirs,
<                             '/scratch/xz/lib')
<             add_dir_to_list(self.compiler.include_dirs,
<                             '/scratch/xz/include')
<             add_dir_to_list(self.compiler.library_dirs,
<                             '/scratch/gdbm/lib')
<             add_dir_to_list(self.compiler.include_dirs,
<                             '/scratch/gdbm/include')
<             add_dir_to_list(self.compiler.library_dirs,
<                             '/scratch/libffi/lib64')
<             add_dir_to_list(self.compiler.include_dirs,
<                             '/scratch/libffi/include')
---
>             add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
>             add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
835,836c813,814
<         #elif self.compiler.find_library_file(lib_dirs, 'ncursesw'):
<         #    curses_library = 'ncursesw'
---
>         elif self.compiler.find_library_file(lib_dirs, 'ncursesw'):
>             curses_library = 'ncursesw'
1152c1130,1136
<         sqlite_inc_paths = [ '/scratch/sqlite/include' ]
---
>         sqlite_inc_paths = [ '/usr/include',
>                              '/usr/include/sqlite',
>                              '/usr/include/sqlite3',
>                              '/usr/local/include',
>                              '/usr/local/include/sqlite',
>                              '/usr/local/include/sqlite3',
>                              ]
1195a1180
>                 os.path.join(sqlite_incdir, '..', 'lib64'),
1196a1182,1183
>                 os.path.join(sqlite_incdir, '..', '..', 'lib64'),
>                 os.path.join(sqlite_incdir, '..', '..', 'lib'),

This is my build process.

mkdir build && cd build

LDFLAGS="-Wl,--rpath=$PREF/glibc/lib -Wl,--rpath=$PREF/bzip2/lib -Wl,--rpath=$PREF/gdbm/lib -Wl,--rpath=$PRE
F/ncurses/lib -Wl,--rpath=$PREF/openssl/lib -Wl,--rpath=$PREF/readline/lib -Wl,--rpath=$PREF/sqlite/lib -Wl,
--rpath=$PREF/xz/lib -Wl,--rpath=$PREF/zlib/lib -Wl,--rpath=$PREF/libffi/lib64 -Wl,--rpath=$PREF/openmpi/lib
 -Wl,--rpath=$PREF/util-linux/lib -Wl,--dynamic-linker=$PREF/glibc/lib/ld-linux-x86-64.so.2" ../configure --
enable-shared --prefix=$PREF/python-latest --with-openssl=$PREF/openssl --enable-optimizations

make -j 8

And this is the error that I get. It's when rebuilding with optimizations.

Rebuilding with profile guided optimizations:
rm -f profile-clean-stamp
make build_all CFLAGS_NODIST=" -fprofile-use -fprofile-correction" LDFLAGS=""
make[1]: Entering directory `/scratch/src/Python-latest/build'
LD_LIBRARY_PATH=/scratch/src/Python-latest/build:/opt/gcc/4.9.2/lib64:/opt/gcc/4.9.2/lib:/scratch/bzip2/lib:/scratch/gdbm/lib:/scratch/libffi/lib64:/scratch/ncurses/lib:/scratch/openmpi/lib:/scratch/openssl/lib:/scratch/readline/lib:/scratch/sqlite/lib:/scratch/util-linux/lib:/scratch/xz/lib:/scratch/zlib/lib:/usr/lib64 ./python -E -S -m sysconfig --generate-posix-vars ;\
        if test $? -ne 0 ; then \
                echo "generate-posix-vars failed" ; \
                rm -f ./pybuilddir.txt ; \
                exit 1 ; \
        fi
Could not import runpy module
Traceback (most recent call last):
  File "/scratch/src/Python-latest/build/../Lib/runpy.py", line 15, in <module>
    import importlib.util
  File "/scratch/src/Python-latest/build/../Lib/importlib/util.py", line 14, in <module>
    from contextlib import contextmanager
  File "/scratch/src/Python-latest/build/../Lib/contextlib.py", line 4, in <module>
    import _collections_abc
SystemError: <built-in function compile> returned NULL without setting an error
generate-posix-vars failed
make[1]: *** [pybuilddir.txt] Error 1
make[1]: Leaving directory `/scratch/src/Python-latest/build'
make: *** [profile-opt] Error 2

I tried searching for the error, and there seem to be similar errors related to improper PYTHONHOME, but I couldn't find anything related to build. I was able to build 3.6 with the same procedure, so I'm not sure if I'm doing something wrong.

It's look you're using a fairly old toolchain. I wouldn't reccomend using fancy modern compiler optimizations with GCC 4.

Hi. Do you mean turning of `--enable-optimizations` or reducing GCC optimization level (O3 etc)?

--enable-optimizations

On Sat, Jul 14, 2018, at 12:31, Jayanth Koushik wrote:
> 
> Jayanth Koushik <jnkoushik@gmail.com> added the comment:
> 
> Hi. Do you mean turning of `--enable-optimizations` or reducing GCC 
> optimization level (O3 etc)?
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue34112>
> _______________________________________

Updating gcc to 8.1.0 fixed the problem.