> I would expect that Py_LIKELY/Py_UNLIKELY helps with this.

Well, PGO is made of many optimizations, not only help branch prediction.

For example, it detects hot code and tries to put functions which call them each other close in memory to enhance the CPU instruction cache efficiency. I tried to annotate "hot" functions but I failed to get reproducible performance results.

commit c6944e7edcacc8960170719623c325aefaf23cac
Author: Victor Stinner <victor.stinner@gmail.com>
Date:   Fri Nov 11 02:13:35 2016 +0100

    Issue #28618: Make hot functions using __attribute__((hot))
    
    When Python is not compiled with PGO, the performance of Python on call_simple
    and call_method microbenchmarks depend highly on the code placement. In the
    worst case, the performance slowdown can be up to 70%.
    
    The GCC __attribute__((hot)) attribute helps to keep hot code close to reduce
    the risk of such major slowdown. This attribute is ignored when Python is
    compiled with PGO.
    
    The following functions are considered as hot according to statistics collected
    by perf record/perf report:
    
    * _PyEval_EvalFrameDefault()
    * call_function()
    * _PyFunction_FastCall()
    * PyFrame_New()
    * frame_dealloc()
    * PyErr_Occurred()

But my change didn't work: "Sadly, it seems like I was just lucky when adding __attribute__((hot)) fixed the issue, because call_method is slow again! (...)  (+51%)"

https://bugs.python.org/issue28618#msg281459

PGO is like black magic for me :-)

Note: Since this change, pyperformance evolved: I removed call_method microbenchmark. It was too instable for various reasons.