http://buildbot.python.org/all/#/builders/47/builds/127

(...)
gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-use -fprofile-correction  -I. -I./Include    -DPy_BUILD_CORE -o Objects/setobject.o Objects/setobject.c
Objects/setobject.c: In function ‘PySet_Clear’:
Objects/setobject.c:2579:1: internal compiler error: in create_edge, at cgraph.c:850
 };
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
Makefile:1584: recipe for target 'Objects/setobject.o' failed
make[1]: Leaving directory '/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build'
make[1]: *** [Objects/setobject.o] Error 1


pythoninfo from the previous build:

* GCC 6.3.0 20170516
* os.uname: posix.uname_result(sysname='Linux', nodename='enable-optimizations-bot', release='4.9.0-3-amd64', version='#1 SMP Debian 4.9.30-2+deb9u2 (2017-06-26)', machine='x86_64')
* platform.platform: Linux-4.9.0-3-amd64-x86_64-with-debian-9.1

http://buildbot.python.org/all/#/builders/47/builds/126/steps/3/logs/stdio

Hi Victor,

My first guess is that the build bot is not cleaning the fprofile
information after updating the source tree.  A few options:

- remove the profile-run-stamp file after checking out new code

- call "make profile-removal" after checkout of new code

- I can try to modify the makefile so that profile-run-stamp depends
  on the source files (.h, .c, etc).  I think that would make it
  re-generate automatically if the sources change.

On 2017-11-06, STINNER Victor wrote:
> 
> New submission from STINNER Victor <victor.stinner@gmail.com>:
> 
> http://buildbot.python.org/all/#/builders/47/builds/127
> 
> (...)
> gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-use -fprofile-correction  -I. -I./Include    -DPy_BUILD_CORE -o Objects/setobject.o Objects/setobject.c
> Objects/setobject.c: In function ‘PySet_Clear’:
> Objects/setobject.c:2579:1: internal compiler error: in create_edge, at cgraph.c:850
>  };
>  ^
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
> Makefile:1584: recipe for target 'Objects/setobject.o' failed
> make[1]: Leaving directory '/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build'
> make[1]: *** [Objects/setobject.o] Error 1
> 
> 
> pythoninfo from the previous build:
> 
> * GCC 6.3.0 20170516
> * os.uname: posix.uname_result(sysname='Linux', nodename='enable-optimizations-bot', release='4.9.0-3-amd64', version='#1 SMP Debian 4.9.30-2+deb9u2 (2017-06-26)', machine='x86_64')
> * platform.platform: Linux-4.9.0-3-amd64-x86_64-with-debian-9.1
> 
> http://buildbot.python.org/all/#/builders/47/builds/126/steps/3/logs/stdio
> 
> ----------
> components: Build
> messages: 305682
> nosy: gregory.p.smith, haypo, nascheme
> priority: normal
> severity: normal
> status: open
> title: AMD64 Debian PGO 3.x buildbot: compilation failed with an internal compiler error in create_edge
> type: performance
> versions: Python 3.7
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue31963>
> _______________________________________

On Mon, Nov 6, 2017 at 3:11 PM Neil Schemenauer <report@bugs.python.org>
wrote:

>
> Neil Schemenauer <nas-python@arctrix.com> added the comment:
>
> Hi Victor,
>
> My first guess is that the build bot is not cleaning the fprofile
> information after updating the source tree.  A few options:
>

But it is.  I actually looked at what the buildbots did before the PR to
change the Makefile.pre.in for profile_opt builds went in.

Here's what happens:

http://buildbot.python.org/all/#/builders/47/builds/127/steps/2/logs/stdio

Notice the "make clean profile-removal" on line 17.

-gps

Oh I see.  I'm at a loss then as to why the build is failing.  A possible clue is the errors like:

profiling:/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Objects/setobject.gcda:Merge mismatch for function 10

A Google search brings up this Stackoverflow page:

https://stackoverflow.com/questions/2590794/gcov-warning-merge-mismatch-for-summaries

To me, that implies that there are compiler outputs (.o, .a, etc) that were build from a previous version source code.  However, the first part of the log shows that the build does "make clean" so I don't see how that is possible.

Is there a way to run "git show 4e38d71a2b7 | git apply -R" on the built host and then try to re-build?

The current "master" branch seems to be building successfully on "AMD64 Debian PGO 3.x".  Can we close this issue?

Same bug, again,  AMD64 Debian PGO 3.x, so I reopen the issue:

http://buildbot.python.org/all/#/builders/47/builds/159

Python/compile.c: In function ‘compiler_sync_comprehension_generator’:
Python/compile.c:5457:1: internal compiler error: in create_edge, at cgraph.c:850
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.

Given this is gcc exploding, I'm not sure there is anything we can do about it.  I _believe_ we're doing everything right.

I'm willing to put some time into trying to fix this, in the case that it is caused by my Makefile changes.  However, it would be very helpful if I could login to the build-bot and try running with the Makefile change backed out.  Is that possible?  Having to setup a VM to match the compiler of the build-bot would take quite a bit a of extra time.

Looking at the "stdout" log file, I can't see that we are doing anything wrong.  Specifically, the ".gc??" files are being removed and so I think that rules out the theory of stale profile information files messing up GCC.

Different error:

http://buildbot.python.org/all/#/builders/47/builds/428

Objects/bytearrayobject.c: In function ‘bytearray_startswith’:
Objects/bytearrayobject.c:2429:1: error: the control flow of function ‘bytearray_startswith’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
 }
 ^
cc1: some warnings being treated as errors

That's mysterious.  I reviewed the 'stdout' log from the buildbot.  The removal of profile data is run early in the build:

find . -name '*.gc??' -exec rm -f {} ';'

Later, the bytearrayobject is compiled with profile generation enabled:

gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_BUILD_CORE -o Objects/bytearrayobject.o Objects/bytearrayobject.c

When the unit tests run, we get errors:

profiling:/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Objects/bytearrayobject.gcda:Merge mismatch for function 35

The second build (with profile use) fails because of the missing profile information:

gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-use -fprofile-correction  -I. -I./Include    -DPy_BUILD_CORE -o Objects/bytearrayobject.o Objects/bytearrayobject.c
Objects/bytearrayobject.c: In function ‘bytearray_startswith’:
Objects/bytearrayobject.c:2429:1: error: the control flow of function ‘bytearray_startswith’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
 }
 ^
cc1: some warnings being treated as errors

Seems like a compiler bug to me.  Wild theory, could it be something to do with address randomization and forking?  Looks like maybe the merge mismatch errors come when the 'python' executable forks.

I didn't see this failure recently, I close the issue.

Hi admins,

I need help for compiling and building Python-3.7.7. 
My system is Ubuntu 16.04 LTS, gcc 5.4.0 20160609

this is the configuration cmd for compling python
./configure --enable-optimizations --enable-shared

when i compiled it, i get these similar error

```
Parser/tokenizer.c: In function ‘PyTokenizer_FindEncodingFilename’:
Parser/tokenizer.c:1909:1: error: the control flow of function ‘PyTokenizer_FindEncodingFilename’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
 }
 ^
Parser/tokenizer.c:1909:1: error: the control flow of function ‘PyTokenizer_FindEncodingFilename’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Parser/tokenizer.c: In function ‘tok_get’:
Parser/tokenizer.c:1909:1: error: the control flow of function ‘tok_get’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Parser/tokenizer.c:1909:1: error: the control flow of function ‘tok_get’ does not match its profile data (counter ‘single’) [-Werror=coverage-mismatch]
Parser/tokenizer.c:1909:1: error: the control flow of function ‘tok_get’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-use -fprofile-correction  -I. -I./Include   -fPIC -DPy_BUILD_CORE -o Objects/bytes_methods.o Objects/bytes_methods.c
Objects/boolobject.c: In function ‘bool_xor’:
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_xor’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
 };
 ^
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_xor’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_xor’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/boolobject.c: In function ‘bool_or’:
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_or’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_or’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_or’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/boolobject.c: In function ‘bool_and’:
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_and’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_and’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_and’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/boolobject.c: In function ‘bool_new’:
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_new’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/boolobject.c:185:1: error: the control flow of function ‘bool_new’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-use -fprofile-correction  -I. -I./Include   -fPIC -DPy_BUILD_CORE -o Objects/bytearrayobject.o Objects/bytearrayobject.c
cc1: some warnings being treated as errors
Makefile:1652: recipe for target 'Objects/boolobject.o' failed
make[1]: *** [Objects/boolobject.o] Error 1
make[1]: *** Waiting for unfinished jobs....
Objects/abstract.c: In function ‘PyObject_GetIter’:
Objects/abstract.c:2642:1: error: the control flow of function ‘PyObject_GetIter’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
 }
 ^
Objects/abstract.c:2642:1: error: the control flow of function ‘PyObject_GetIter’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PyObject_GetIter’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/abstract.c: In function ‘PySequence_InPlaceRepeat’:
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_InPlaceRepeat’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_InPlaceRepeat’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_InPlaceRepeat’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/abstract.c: In function ‘PySequence_InPlaceConcat’:
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_InPlaceConcat’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_InPlaceConcat’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_InPlaceConcat’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/abstract.c: In function ‘PySequence_Repeat’:
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_Repeat’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_Repeat’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_Repeat’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/abstract.c: In function ‘PySequence_Concat’:
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_Concat’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_Concat’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PySequence_Concat’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/abstract.c: In function ‘PyObject_CopyData’:
Objects/abstract.c:2642:1: error: the control flow of function ‘PyObject_CopyData’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PyObject_CopyData’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
Objects/abstract.c: In function ‘PyObject_LengthHint’:
Objects/abstract.c:2642:1: error: the control flow of function ‘PyObject_LengthHint’ does not match its profile data (counter ‘arcs’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PyObject_LengthHint’ does not match its profile data (counter ‘indirect_call’) [-Werror=coverage-mismatch]
Objects/abstract.c:2642:1: error: the control flow of function ‘PyObject_LengthHint’ does not match its profile data (counter ‘time_profiler’) [-Werror=coverage-mismatch]
cc1: some warnings being treated as errors
Makefile:1652: recipe for target 'Parser/tokenizer.o' failed
make[1]: *** [Parser/tokenizer.o] Error 1
cc1: some warnings being treated as errors
Makefile:1652: recipe for target 'Objects/abstract.o' failed
make[1]: *** [Objects/abstract.o] Error 1
make[1]: Leaving directory '/home/ion/Documents/Python-3.7.7'
Makefile:534: recipe for target 'profile-opt' failed
make: *** [profile-opt] Error 2
```

what should i do to fix this? thank you very much.

best regards,
Albert Christianto

> I need help for compiling and building Python-3.7.7. My system is Ubuntu 16.04 LTS, gcc 5.4.0 20160609

You have different options:

* Upgrade Ubuntu to the newer LTS to get a more recent C compiler
* Don't use PGO

Also, you might use a pre-built package rather than build Python yourself? See https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa

Sorry for my late response. 
Well, thank you very much for your fast response to help me. 
Actually, I have solved the problem in 3 hours after I posted my question. hehehe.
I found this tip about uncleaned files after executing "make clean" (https://bugs.python.org/msg346553). So, I decided to wipe out all python3.7 files from my computer and repeat the installation process. It worked!!
Anyway, thank you again for your response and your tip. It really made my day like I am not alone while doing some python program debugging. (^v^)
best regards,
Albert Christianto