> -qmaxmem affects the compiler optimization.

I know, that's just from the Python README.AIX.  I didn't expect it to have any influence.


> -Wl,-bmaxdata:0x800000000 is a GCC command line option.

That is indeed surprising. Linking is prepared by a script:

$ ./Modules/makexp_aix Modules/python.exp . libpython3.10.a;  xlc -L/usr/lib64 -q64 -Wl,-bmaxdata:0x800000000    -Wl,-bE:Modules/python.exp -lld -o python Programs/python.o libpython3.10.a -lintl -ldl  -lpthread -lm   -lm

The xlc command runs without warnings or errors.



Without Wl,-bmaxdata:0x800000000
================================

$ ./python -m test -uall test_decimal
0:00:00 Run tests sequentially
0:00:00 [1/1] test_decimal
Killed


With Wl,-bmaxdata:0x800000000
=============================

$ ./python -m test -uall test_decimal
0:00:00 Run tests sequentially
0:00:00 [1/1] test_decimal

== Tests result: SUCCESS ==

1 test OK.

Total duration: 17.3 sec
Tests result: SUCCESS


> On the other hand, -bmaxdata behaves differently in 32 bit mode and 64 bit mode.  In 32 bit mode, it increases the heap size from the default 256MB.  In 64 bit mode, it sets a guaranteed maximum size for the heap.

Yes, that's what I expected. The test only allocates that much memory
for 64-bit builds.  The workaround only needs to be enabled for 64-bit.

So a memory softlimit, same as e.g. djb uses for qmail with his
softlimit program, is exactly what I was looking for.


> I sort of think that the solution desired for the testcase is PSALLOC=early to match traditional Unix/Linux malloc() behavior.

I can try that, but our test suite might complain about the environment
being modified.