Calling this a compile error - as it seems to be compiler dependent.

In other projects - when I have experienced issues as this it has been an uninitiated variable - somewhere.

I would appreciate some suggestions on how to best debug this - as it seems to occur even before tracemalloc can be activated.

$ ./python -X tracemalloc
Objects/genobject.c:127: _PyObject_GC_TRACK: Assertion "!(((PyGC_Head *)(op)-1)->_gc_next != 0)" failed: object already tracked by the garbage collector
Enable tracemalloc to get the memory block allocation traceback

object address  : 30085150
object refcount : 0
object type     : 20013ea0
object type name: generator
object repr     : <refcnt 0 at 30085150>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: core initialized

Current thread 0x00000001 (most recent call first):
  File "<frozen importlib._bootstrap_external>", line 1593 in _setup
  File "<frozen importlib._bootstrap_external>", line 1634 in _install
  File "<frozen importlib._bootstrap>", line 1189 in _install_external_importers
IOT/Abort trap(coredump)

p.s. I can always build using a different compiler and try to get it to report on this object using the values listed above - and/or insert more debug code.

Next step I'll try is using dbx (AIX debugger) for a stacktrace.

Thanks!

dbx output:

Again: help appreciated.

(dbx) run  -X tracemalloc
Objects/genobject.c:127: _PyObject_GC_TRACK: Assertion "!(((PyGC_Head *)(op)-1)->_gc_next != 0)" failed: object already tracked by the garbage collector
Enable tracemalloc to get the memory block allocation traceback

object address  : 30085150
object refcount : 0
object type     : 20014188
object type name: generator
object repr     : <refcnt 0 at 30085150>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: core initialized

Current thread 0x00000001 (most recent call first):
  File "<frozen importlib._bootstrap_external>", line 1593 in _setup
  File "<frozen importlib._bootstrap_external>", line 1634 in _install
  File "<frozen importlib._bootstrap>", line 1189 in _install_external_importers

IOT/Abort trap in pthread_kill at 0xd057a02c ($t1)
0xd057a02c (pthread_kill+0xac) 80410014            lwz   r2,0x14(r1)
(dbx) where
pthread_kill(??, ??) at 0xd057a02c
_p_raise(??) at 0xd0579408
raise.raise(??) at 0xd0123344
abort() at 0xd0189918
IPRA.$fatal_error(stream = 0x2006e370, header = 805397344, prefix = (nil), msg = (nil), status = 0), line 2183 in "pylifecycle.c"
_Py_FatalErrorFunc(func = "", msg = "\n"), line 2283 in "pylifecycle.c"
_PyObject_AssertFailed(obj = 0x30086038, expr = "", msg = (nil), file = "\177\200", line = 537322352, function = ""), line 2195 in "object.c"
gen_dealloc(gen = 0x2004af44), line 65532 in "pycore_object.h"
_Py_Dealloc(op = 0x00000013), line 2206 in "object.c"
unnamed block in _PyEval_EvalFrameDefault(tstate = 0x2006e370, f = 0x30023b0c, throwflag = 804397120), line 1046 in "object.h"
_PyEval_EvalFrameDefault(tstate = 0x2006e370, f = 0x30023b0c, throwflag = 804397120), line 1046 in "object.h"
unnamed block in IPRA.$function_code_fastcall(tstate = 0x2004af44, co = 0x20030848, args = 0x30066f7c, nargs = 805735610, globals = 0x30068cb8), line 40 in "pycore_ceval.h"
IPRA.$function_code_fastcall(tstate = 0x2004af44, co = 0x20030848, args = 0x30066f7c, nargs = 805735610, globals = 0x30068cb8), line 40 in "pycore_ceval.h"
_PyFunction_Vectorcall(func = 0x00000006, stack = 0x30083898, nargsf = 804397328, kwnames = 0x3006e110), line 366 in "call.c"
unnamed block in _PyEval_EvalFrameDefault(tstate = 0x2004af44, f = 0xffffffe2, throwflag = 804397584), line 739 in "abstract.h"
_PyEval_EvalFrameDefault(tstate = 0x2004af44, f = 0xffffffe2, throwflag = 804397584), line 739 in "abstract.h"
unnamed block in IPRA.$function_code_fastcall(tstate = 0x2004af44, co = 0x20030848, args = 0x30023b14, nargs = 805682254, globals = 0x3005bc38), line 40 in "pycore_ceval.h"
IPRA.$function_code_fastcall(tstate = 0x2004af44, co = 0x20030848, args = 0x30023b14, nargs = 805682254, globals = 0x3005bc38), line 40 in "pycore_ceval.h"
_PyFunction_Vectorcall(func = 0x00000005, stack = 0x3005da78, nargsf = 804397856, kwnames = 0xdeadbeef), line 366 in "call.c"
unnamed block in _PyEval_EvalFrameDefault(tstate = 0x100fbeb8, f = 0x2006e370, throwflag = 804398128), line 739 in "abstract.h"
_PyEval_EvalFrameDefault(tstate = 0x100fbeb8, f = 0x2006e370, throwflag = 804398128), line 739 in "abstract.h"
unnamed block in IPRA.$function_code_fastcall(tstate = 0x10073f50, co = 0x2000a350, args = 0x20011a58, nargs = 537167132, globals = 0x30049118), line 40 in "pycore_ceval.h"
IPRA.$function_code_fastcall(tstate = 0x10073f50, co = 0x2000a350, args = 0x20011a58, nargs = 537167132, globals = 0x30049118), line 40 in "pycore_ceval.h"
_PyFunction_Vectorcall(func = 0xd98069bf, stack = 0x00000008, nargsf = 4029698048, kwnames = 0x2006e370), line 366 in "call.c"
IPRA.$_PyObject_CallFunctionVa(tstate = 0x100f6cb0, callable = (nil), format = "/\362%\340-zj\244^P^O\242\234/\362&\340^\220K\250 ^D\205^\^PCl\243/\362&\340", va = "", is_size_t = 272852043), line 62 in "abstract.h"
IPRA.$callmethod(tstate = 0x100fa29c, callable = 0x2ff226e0, format = (invalid char ptr (0x5e904ba8)), va = "", is_size_t = 272854179), line 614 in "call.c"
PyObject_CallMethod(obj = 0x102c266c, name = "", format = "", ... = 0x20077748, 0xffff, 0x20077748, 0x20, 0x10), line 634 in "call.c"
init_importlib_external(tstate = 0xdeadbeef), line 209 in "pylifecycle.c"
IPRA.$init_interp_main(tstate = 0x00000001), line 993 in "pylifecycle.c"
pyinit_main(tstate = 0x2003c984), line 1097 in "pylifecycle.c"
Py_InitializeFromConfig(config = 0xf078b380), line 1141 in "pylifecycle.c"
pymain_init(args = 0x00000001), line 66 in "main.c"
pymain_main(args = 0x00000003), line 653 in "main.c"
Py_BytesMain(argc = -559038737, argv = 0xdeadbeef), line 686 in "main.c"
python.main(argc = 0, argv = (nil)), line 16 in "python.c"

Since I just had a similar incident with the Intel compiler, I just
want to point out that compiling CPython still requires -fwrapv.

I've mentioned this before in AIX issues, and no one has pointed
out the corresponding xlc flag.


https://gcc.gnu.org/legacy-ml/gcc/2007-01/msg00062.html

"Dan Berlin says that xlc assumes signed overflow never occurs"

The assertion failure occurs in _PyObject_GC_TRACK() at:

static void
gen_dealloc(PyGenObject *gen)
{
    PyObject *self = (PyObject *) gen;

    _PyObject_GC_UNTRACK(gen);

    if (gen->gi_weakreflist != NULL)
        PyObject_ClearWeakRefs(self);

    _PyObject_GC_TRACK(self);  // <==== HERE

    ...
}

It's surprising that the generator is still tracked by the GC after _PyObject_GC_UNTRACK().


> Calling this a compile error - as it seems to be compiler dependent.

Do you reproduce the bug if you build Python with GCC?

Which ./configure command did you use? What are the compiler and linker flags?

You can try:

./python -m test.pythoninfo|grep -E 'CFLAGS|CC|OPT|LDFLAGS'

After git bisect - comes down to:

Bisecting: 0 revisions left to test after this (roughly 0 steps)
[0003c2dc1d4cf5b122e73e83177fd274fa9a9913] bpo-40096: Support __attribute__((__noreturn__)) on xlc (GH-19204)

It is only XLC-v16 that fails. XLC-v11 and XLC-v13 work fine. Am digging to see which version (< v16, or >= v16) is not working as expected.

On 10/04/2020 14:01, STINNER Victor wrote:
> STINNER Victor <vstinner@python.org> added the comment:
>
> The assertion failure occurs in _PyObject_GC_TRACK() at:
>
> static void
> gen_dealloc(PyGenObject *gen)
> {
>     PyObject *self = (PyObject *) gen;
>
>     _PyObject_GC_UNTRACK(gen);
>
>     if (gen->gi_weakreflist != NULL)
>         PyObject_ClearWeakRefs(self);
>
>     _PyObject_GC_TRACK(self);  // <==== HERE
>
>     ...
> }
>
> It's surprising that the generator is still tracked by the GC after _PyObject_GC_UNTRACK().
>
>
>> Calling this a compile error - as it seems to be compiler dependent.
> Do you reproduce the bug if you build Python with GCC?

To be clear - gcc does not not have an issue. As I stated elsewhere - it
is specific to xlc-v16, so likely it is a compiler error.

See also the result of `git bisect` study.

>
> Which ./configure command did you use? What are the compiler and linker flags?
>
> You can try:
>
> ./python -m test.pythoninfo|grep -E 'CFLAGS|CC|OPT|LDFLAGS'

With:

$ ./python -m test.pythoninfo|grep -E 'CFLAGS|CC|OPT|LDFLAGS'
Objects/genobject.c:127: _PyObject_GC_TRACK: Assertion "!(((PyGC_Head
*)(op)-1)->_gc_next != 0)" failed: object already tracked by the garbage
collector
Enable tracemalloc to get the memory block allocation traceback

object address  : 30084150
object refcount : 0
object type     : 200144a8
object type name: generator
object repr     : <refcnt 0 at 30084150>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: core initialized

Current thread 0x00000001 (most recent call first):
  File "<frozen importlib._bootstrap_external>", line 1593 in _setup
  File "<frozen importlib._bootstrap_external>", line 1634 in _install
  File "<frozen importlib._bootstrap>", line 1189 in
_install_external_importers
ksh: 27328848 IOT/Abort trap(coredump)

Without error:

$ ./python -m test.pythoninfo|grep -E 'CFLAGS|CC|OPT|LDFLAGS'
os.environ[CC]: xlc_r
sysconfig[CC]: xlc_r
sysconfig[CFLAGS]: -O
sysconfig[CONFIG_ARGS]: '--with-openssl=/opt/aixtools'
'--without-computed-gotos' '--with-pydebug' 'CC=xlc_r'
sysconfig[OPT]: -O
sysconfig[PY_CFLAGS]: -O
sysconfig[PY_CFLAGS_NODIST]: -I./Include/internal
sysconfig[PY_STDMODULE_CFLAGS]: -O -I./Include/internal -I. -I./Include

> ----------
> nosy: +vstinner
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40244>
> _______________________________________
>

> [0003c2dc1d4cf5b122e73e83177fd274fa9a9913] bpo-40096: Support __attribute__((__noreturn__)) on xlc (GH-19204)

The fix looks simple: revert this change, no?

I've just compiled python with (xlc 16.1.0, debug build) and can't experience that compile failure, can you give a specific spot that failure happens?

-bash-4.4$ ./python -m test.pythoninfo|grep -E 'CFLAGS|CC|OPT|LDFLAGS'
sysconfig[CC]: /opt/IBM/xlc/16.1.0/bin/xlc_r
sysconfig[CFLAGS]: -DNDEBUG -O
sysconfig[CONFIG_ARGS]: 'CC=/opt/IBM/xlc/16.1.0/bin/xlc_r'
sysconfig[OPT]: -DNDEBUG -O
sysconfig[PY_CFLAGS]: -DNDEBUG -O
sysconfig[PY_CFLAGS_NODIST]: -I./Include/internal
sysconfig[PY_STDMODULE_CFLAGS]: -DNDEBUG -O -I./Include/internal -I. -I./Include

After my build I have the following:

$ ./python -m test.pythoninfo|grep -E 'CFLAGS|CC|OPT|LDFLAGS'
Objects/genobject.c:127: _PyObject_GC_TRACK: Assertion "!(((PyGC_Head
*)(op)-1)->_gc_next != 0)" failed: object already tracked by the garbage
collector
Enable tracemalloc to get the memory block allocation traceback

object address  : 30084150
object refcount : 0
object type     : 20013710
object type name: generator
object repr     : <refcnt 0 at 30084150>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: core initialized

Current thread 0x00000001 (most recent call first):
  File "<frozen importlib._bootstrap_external>", line 1593 in _setup
  File "<frozen importlib._bootstrap_external>", line 1634 in _install
  File "<frozen importlib._bootstrap>", line 1189 in
_install_external_importers
ksh: 22151670 IOT/Abort trap(coredump)

See attached file for the script capture.

On 11/04/2020 15:13, Batuhan Taskaya wrote:
> Batuhan Taskaya <batuhanosmantaskaya@gmail.com> added the comment:
>
> I've just compiled python with (xlc 16.1.0, debug build) and can't experience that compile failure, can you give a specific spot that failure happens?
>
> -bash-4.4$ ./python -m test.pythoninfo|grep -E 'CFLAGS|CC|OPT|LDFLAGS'
> sysconfig[CC]: /opt/IBM/xlc/16.1.0/bin/xlc_r
> sysconfig[CFLAGS]: -DNDEBUG -O
> sysconfig[CONFIG_ARGS]: 'CC=/opt/IBM/xlc/16.1.0/bin/xlc_r'
> sysconfig[OPT]: -DNDEBUG -O
> sysconfig[PY_CFLAGS]: -DNDEBUG -O
> sysconfig[PY_CFLAGS_NODIST]: -I./Include/internal
> sysconfig[PY_STDMODULE_CFLAGS]: -DNDEBUG -O -I./Include/internal -I. -I./Include
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40244>
> _______________________________________
>

Oh, looks like the problem is --without-computed-gotos, I just tried 2 times one is with --without-computed-gotos and one is not. I could successfully reproduce the problem with it. I'll continue triaging.

@Michael.Felt can you just insert some prints between these 3 to see what is going on (and re-compile)

static void
gen_dealloc(PyGenObject *gen)
{
    PyObject *self = (PyObject *) gen;
<<<<
    _PyObject_GC_UNTRACK(gen);
<<<<
    if (gen->gi_weakreflist != NULL)
        PyObject_ClearWeakRefs(self);
<<<<
    _PyObject_GC_TRACK(self);


something like this should work: printf("%ld\n", _Py_AS_GC(self)->_gc_next);

By the noticed something that looks kind of weird, not sure if it is intentional or not but shouldn't this be self instead of gen

static void
gen_dealloc(PyGenObject *gen)
{
    PyObject *self = (PyObject *) gen;
<<<<
_PyObject_GC_UNTRACK(gen);
<<<<
    if (gen->gi_weakreflist != NULL)
        PyObject_ClearWeakRefs(self);
    _PyObject_GC_TRACK(self);

With the print statements - it does not crash:

         ./python -E -S -m sysconfig --generate-posix-vars ; if test $?
-ne 0 ; then  echo "generate-posix-vars failed" ;  rm -f
./pybuilddir.txt ;  exit 1 ;  fi
Objects/genobject.c:122:537318120
Objects/genobject.c:126:0
Objects/genobject.c:131:0
Objects/genobject.c:135:537318120
Objects/genobject.c:122:805942488
Objects/genobject.c:126:0
Objects/genobject.c:131:0
Objects/genobject.c:135:537318120
...

diff --git a/Objects/genobject.c b/Objects/genobject.c
index d3455f8..b8287a3 100644
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -117,14 +117,23 @@ exc_state_clear(_PyErr_StackItem *exc_state)
 static void
 gen_dealloc(PyGenObject *gen)
 {
+#include <stdio.h>
     PyObject *self = (PyObject *) gen;
+    fprintf(stderr,"%s:%d:%ld\n", __FILE__,__LINE__,
_Py_AS_GC(self)->_gc_next);
+    fflush(stderr);

     _PyObject_GC_UNTRACK(gen);
+    fprintf(stderr,"%s:%d:%ld\n", __FILE__,__LINE__,
_Py_AS_GC(self)->_gc_next);
+    fflush(stderr);

     if (gen->gi_weakreflist != NULL)
         PyObject_ClearWeakRefs(self);
+    fprintf(stderr,"%s:%d:%ld\n", __FILE__,__LINE__,
_Py_AS_GC(self)->_gc_next);
+    fflush(stderr);

     _PyObject_GC_TRACK(self);
+    fprintf(stderr,"%s:%d:%ld\n", __FILE__,__LINE__,
_Py_AS_GC(self)->_gc_next);
+    fflush(stderr);

     if (PyObject_CallFinalizerFromDealloc(self))
         return;                     /* resurrected.  :( */
$

On 13/04/2020 23:19, Batuhan Taskaya wrote:
> Batuhan Taskaya <batuhanosmantaskaya@gmail.com> added the comment:
>
> @Michael.Felt can you just insert some prints between these 3 to see what is going on (and re-compile)
>
> static void
> gen_dealloc(PyGenObject *gen)
> {
>     PyObject *self = (PyObject *) gen;
> <<<<
>     _PyObject_GC_UNTRACK(gen);
> <<<<
>     if (gen->gi_weakreflist != NULL)
>         PyObject_ClearWeakRefs(self);
> <<<<
>     _PyObject_GC_TRACK(self);
>
>
> something like this should work: printf("%ld\n", _Py_AS_GC(self)->_gc_next);
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40244>
> _______________________________________
>

Also tried this:

         ./python -E -S -m sysconfig --generate-posix-vars ; if test $?
-ne 0 ; then  echo "generate-posix-vars failed" ;  rm -f
./pybuilddir.txt ;  exit 1 ;  fi
Objects/genobject.c:127: _PyObject_GC_TRACK: Assertion "!(((PyGC_Head
*)(op)-1)->_gc_next != 0)" failed: object already tracked by the garbage
collector
Enable tracemalloc to get the memory block allocation traceback

object address  : 30084150
object refcount : 0
object type     : 20013718
object type name: generator
object repr     : <refcnt 0 at 30084150>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: core initialized

Current thread 0x00000001 (most recent call first):
  File "<frozen importlib._bootstrap_external>", line 1593 in _setup
  File "<frozen importlib._bootstrap_external>", line 1634 in _install
  File "<frozen importlib._bootstrap>", line 1189 in
_install_external_importers
/bin/sh: 17302202 IOT/Abort trap(coredump)
make: 1254-004 The error code from the last command is 134.

Stop.
$ git diff
diff --git a/Objects/genobject.c b/Objects/genobject.c
index d3455f8..e783636 100644
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -119,7 +119,7 @@ gen_dealloc(PyGenObject *gen)
 {
     PyObject *self = (PyObject *) gen;

-    _PyObject_GC_UNTRACK(gen);
+    _PyObject_GC_UNTRACK(self);

     if (gen->gi_weakreflist != NULL)
         PyObject_ClearWeakRefs(self);
$

On 14/04/2020 00:18, Batuhan Taskaya wrote:
> Batuhan Taskaya <batuhanosmantaskaya@gmail.com> added the comment:
>
> By the noticed something that looks kind of weird, not sure if it is intentional or not but shouldn't this be self instead of gen
>
> static void
> gen_dealloc(PyGenObject *gen)
> {
>     PyObject *self = (PyObject *) gen;
> <<<<
> _PyObject_GC_UNTRACK(gen);
> <<<<
>     if (gen->gi_weakreflist != NULL)
>         PyObject_ClearWeakRefs(self);
>     _PyObject_GC_TRACK(self);
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40244>
> _______________________________________
>

> With the print statements - it does not crash:

I think this isn't directly relevant with prints but about re-compiling? (just guessing). Because I experienced when I compile python for the first time on a clean AIX environment, all extension modules failed to build. When I recompiled (with keeping all artifacts from previous build) some of them successfully got compiled. When I try to compile again, most of them successfully compiled. I'm sorry but I don't know why this happens or how to solve it. We can always revert that change but I guess that isn't the real problem.

> Also tried this:

Thanks for it, that was just something that I slightly suspected but expected to see it crash.

On 14/04/2020 14:54, Batuhan Taskaya wrote:
> Batuhan Taskaya <batuhanosmantaskaya@gmail.com> added the comment:
>
>> With the print statements - it does not crash:
> I think this isn't directly relevant with prints but about re-compiling? (just guessing).

I only recompiled the one .c file. With that one file re-compiled -
wqith fprintf statements it succeeds, restore the original .c file (git
checkout -- Objects/whatever.c; make - it fails.

Tomorrow I'll search for the option(s) needed to get (complete) assembly
code listing and try to see (and understand) the difference between what
xlc-v13 and xlc-v16 makes. And, what I shall also test - is to recompile
only this one file using xlc-v13 and see if the make then proceeds normally.

>  Because I experienced when I compile python for the first time on a clean AIX environment, all extension modules failed to build.

I only see this happen (on occasion) when I use make -j4 (or greater) -
and I have seen it happen to a lessor extent with -j2. On the subsequent
passes, whatever it is that setup.py (guessing) really needs is now
available - and the modules build as expected.

This is also why, for the last 4 years I have used my own personal
server - where I control everything (mainly NO other party OSS packaged
software and their artifacts).

>  When I recompiled (with keeping all artifacts from previous build) some of them successfully got compiled. When I try to compile again, most of them successfully compiled. I'm sorry but I don't know why this happens or how to solve it.
why - I do not understand the finer details either, but my guess is that
it is related to linking. I am nearly "amazed" - yet happy - that the
PPC64 AIX 3.X compile succeeds - but acknowledge the 22K+ lines of
warnings is related to the over parallelization of the linking.
> We can always revert that change but I guess that isn't the real problem.

No. I do not think it is the real problem either. And I do not know
compiler behavior well enough. Actually, considering the setting is
still -O0 (aka no optimization) I am surprised it has any effect. if I
understood correctly "no return" is intended to help the optimizer make
"informed" decisions.

As Victor commented earlier - very much looking like a compiler bug.
That said, still do not know what to say/write to software support as a
complaint.

>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40244>
> _______________________________________
>

On 14/04/2020 19:28, Michael Felt wrote:
> Michael Felt <aixtools@felt.demon.nl> added the comment:
>
> On 14/04/2020 14:54, Batuhan Taskaya wrote:
>> Batuhan Taskaya <batuhanosmantaskaya@gmail.com> added the comment:
>>
>>> With the print statements - it does not crash:
>> I think this isn't directly relevant with prints but about re-compiling? (just guessing).
> I only recompiled the one .c file. With that one file re-compiled -
> wqith fprintf statements it succeeds, restore the original .c file (git
> checkout -- Objects/whatever.c; make - it fails.
>
> Tomorrow I'll search for the option(s) needed to get (complete) assembly
> code listing and try to see (and understand) the difference between what
> xlc-v13 and xlc-v16 makes. And, what I shall also test - is to recompile
> only this one file using xlc-v13 and see if the make then proceeds normally.
>
>
Many pages of output - and I confess - I do have some difficulty reading
code every now and then.

As the "bug" wherever it may be is related, I am guessing, to compiler
optimization and how to deal with routines with "no return".

Trying to understand the listings - I ran across:

./Include/object.h:typedef void (*destructor)(PyObject *);

Could the error be related to compilers confusing a routine with no
return, versus a routine returning a pointer to a "void"?

recall the code:

static void
gen_dealloc(PyGenObject *gen)

Comments?

Michael

> As Victor commented earlier - very much looking like a compiler bug.
> That said, still do not know what to say/write to software support as a
> complaint.
>
>> ----------
>>
>> _______________________________________
>> Python tracker <report@bugs.python.org>
>> <https://bugs.python.org/issue40244>
>> _______________________________________
>>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40244>
> _______________________________________
>

On 15/04/2020 11:03, Michael Felt wrote:
> Michael Felt <aixtools@felt.demon.nl> added the comment:
>
> On 14/04/2020 19:28, Michael Felt wrote:
>> Michael Felt <aixtools@felt.demon.nl> added the comment:
>>
>> On 14/04/2020 14:54, Batuhan Taskaya wrote:
>>> Batuhan Taskaya <batuhanosmantaskaya@gmail.com> added the comment:
>>>
>>>> With the print statements - it does not crash:
>>> I think this isn't directly relevant with prints but about re-compiling? (just guessing).
>> I only recompiled the one .c file. With that one file re-compiled -
>> wqith fprintf statements it succeeds, restore the original .c file (git
>> checkout -- Objects/whatever.c; make - it fails.
>>
>> Tomorrow I'll search for the option(s) needed to get (complete) assembly
>> code listing and try to see (and understand) the difference between what
>> xlc-v13 and xlc-v16 makes. And, what I shall also test - is to recompile
>> only this one file using xlc-v13 and see if the make then proceeds normally.
>>
>>
> Many pages of output - and I confess - I do have some difficulty reading
> code every now and then.
>
> As the "bug" wherever it may be is related, I am guessing, to compiler
> optimization and how to deal with routines with "no return".
>
> Trying to understand the listings - I ran across:
>
> ./Include/object.h:typedef void (*destructor)(PyObject *);
>
> Could the error be related to compilers confusing a routine with no
> return, versus a routine returning a pointer to a "void"?

Although I as I think about it I believe the statement is correct:
"destructor" is a typedef of a pointer to a function (with an argument
of PyObject *) that has no_return (aka == void).

Sigh.

Instead. The two listings: note, they are practically identical for the
first 60 pages (aka skip to 'Page 61')

>
> recall the code:
>
> static void
> gen_dealloc(PyGenObject *gen)
>
> Comments?
>
> Michael
>
>> As Victor commented earlier - very much looking like a compiler bug.
>> That said, still do not know what to say/write to software support as a
>> complaint.
>>
>>> ----------
>>>
>>> _______________________________________
>>> Python tracker <report@bugs.python.org>
>>> <https://bugs.python.org/issue40244>
>>> _______________________________________
>>>
>> ----------
>>
>> _______________________________________
>> Python tracker <report@bugs.python.org>
>> <https://bugs.python.org/issue40244>
>> _______________________________________
>>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40244>
> _______________________________________
>

> No. I do not think it is the real problem either. And I do not know
compiler behavior well enough. Actually, considering the setting is
still -O0 (aka no optimization) I am surprised it has any effect. if I
understood correctly "no return" is intended to help the optimizer make
"informed" decisions.

Does removing all no returns change anything for you? (It didn't change anything for me, if I did it correctly)
find ./ -name "*.c" -type f -exec perl -pi -e 's/_Py_NO_RETURN//g' '{}' \;

Moving assertion from _PyObject_GC_TRACK to gen_dealloc (just before the _PyObject_GC_TRACK call) results with success (????)

     if (gen->gi_weakreflist != NULL)
         PyObject_ClearWeakRefs(self);
-
+    _PyObject_ASSERT_FROM(self, !_PyObject_GC_IS_TRACKED(self),
+                          "object already tracked by they garbage collector",
+                          __FILE__, __LINE__, "_PyObject_GC_TRACK");
     _PyObject_GC_TRACK(self);
 
     if (PyObject_CallFinalizerFromDealloc(self))

When I have more time I hope to investigate specifically what is
different in the assembly code - with/without the __noreturn__ change.

On 19/04/2020 08:20, Batuhan Taskaya wrote:
> Batuhan Taskaya <batuhanosmantaskaya@gmail.com> added the comment:
>
> Moving assertion from _PyObject_GC_TRACK to gen_dealloc (just before the _PyObject_GC_TRACK call) results with success (????)
>
>      if (gen->gi_weakreflist != NULL)
>          PyObject_ClearWeakRefs(self);
> -
> +    _PyObject_ASSERT_FROM(self, !_PyObject_GC_IS_TRACKED(self),
> +                          "object already tracked by they garbage collector",
> +                          __FILE__, __LINE__, "_PyObject_GC_TRACK");
>      _PyObject_GC_TRACK(self);
>  
>      if (PyObject_CallFinalizerFromDealloc(self))
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40244>
> _______________________________________
>

Any news about this? Should we revert the commit for 3.9?

I think this is showing up again. Ot seemed to be away when using xlcv13 (and is away with xlcv11).

I switched my bot off of xlc (v13) because compile fails again - and I'll investigate manually using xlc again.

If you could prep a PR where the change is reverted - I would appreciate it. If no time, I'll get to it as soon as I can.

> If you could prep a PR where the change is reverted - I would appreciate it. If no time, I'll get to it as soon as I can.

I see. I'll try to get a patch to deactivate it and add a comment (for future about this issue).

New changeset 033d10bd21d962a59c6c4fc503092046baa451a1 by Batuhan Taskaya in branch 'master':
bpo-40244: Remove XLC's support from the noreturn flag (GH-20588)
https://github.com/python/cpython/commit/033d10bd21d962a59c6c4fc503092046baa451a1

New changeset 50e847a9eb03f59e1d9268e46f3f98c2679caebd by Miss Islington (bot) in branch '3.9':
bpo-40244: Remove XLC's support from the noreturn flag (GH-20588)
https://github.com/python/cpython/commit/50e847a9eb03f59e1d9268e46f3f98c2679caebd