On AIX, executables are compiled by default so that they cannot allocate more than 256MB of memory.

This is not enough in some cases; for example this is not enough to get the Python test suite to run completely.

As explained in IBM documentation:

By default each program gets one segment register (see 2.24) for its
data segment. As each segment register covers 256 MB, any calls to
malloc more will fail. Also programs that declare large global or static
arrays may fail to load. To allocate more segment registers to your
program, use the linker option -bmaxdata to specify the number of bytes
you need in the data segment as follows:

cc -o myprog -bmaxdata:0x20000000 myprog.c

The example above would allocate an additional segment register to allow
for 512MB of data.

export LDR_CNTRL=MAXDATA=0x20000000
start_process
unset LDR_CNTRL

Marking An Executable For Large Page Use
The XCOFF header in an executable file contains a new flag to indicate that the program wants to use
large pages to back its data and heap segments. This flag can be set when the application is linked by
specifying the -blpdata option on the ld command. The flag can also be set or cleared using the ldedit
command. The "ldedit -blpdata filename" command sets the large page data/heap flag in the specified
file. The "ldedit -bnolpdata filename" clears the large page flag. The ldedit command may also be used
to set an executable's maxdata value

Here is a patch that would provide an --enable-aix-maxdata option to configure, in order to support 512MB of mermory.

I have not tested it yet.

I don't see why we would provide an option to replace a magic number with another magic number. I find it strange that AIX doesn't simply enable unbounded virtual address spaces.
(by "unbounded" I mean "bounded by CPU limitations only")

Yes, welcome into the strange world of AIX...

You can allow up to 8 segments, which represent 2GB:
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=/com.ibm.itame3.doc_5.1/am51_perftune113.htm

We could change the option to become:
--with-aix-maxdata=value
where value could be between 0 and 8, with a default at 0 (nothing passed to the compiler).

Would that be OK with you?

The good news is that thanks to this patch I was able to play the full python test suite on AIX:

313 tests OK.
8 tests failed:
    test_asyncore test_cmath test_distutils test_fileio test_locale
    test_resource test_time test_wait4
16 tests skipped:
    test_ctypes test_curses test_dbm_gnu test_epoll test_gdb
    test_kqueue test_largefile test_ossaudiodev test_pep277
    test_startfile test_tk test_ttk_guionly test_unicode_file
    test_winreg test_winsound test_zipfile64

Not too bad.

> We could change the option to become:
> --with-aix-maxdata=value
> where value could be between 0 and 8, with a default at 0 (nothing
> passed to the compiler).

Doesn't LDFLAGS work for that? I don't think we want a lot of
platform-specific options like that.

This link flag should only be applied to the python executable.
There is already LINKFORSHARED which is great for that.
LDFLAGS will apply it to all the shared libraries which is not really what we want.

I can compile Python with support for 512MB without any modification to configure like that:

CC=xlc_r OPT="-O2 -qmaxmem=100000" ./configure --without-computed-gotos --enable-shared
make LINKFORSHARED="-Wl,-bE:Modules/python.exp -lld -Wl,-bmaxdata:0x20000000"

It is just not very elegant and will make the builbdot script even more complex than it is currently.
Also it will make it more difficult for AIX users to find out about this limitation or build python with support for more memory (though I agree this represents very few people).

If you don't want it in configure though, I will add a note in AIX-NOTES about this limitation and how to build Python with support for more memory.

> LDFLAGS will apply it to all the shared libraries which is not really
> what we want.

Is there a downside?

I don't know distutils well enough, but I am afraid the LDFLAGS flag used to compile python is reused in some way when linking some python extensions or some application that embed python.
(I use an application that embed python on AIX, I should check what happens when I compile it).

Except for that, I can't see any problem now, but I would not be feel comfortable propagating this option in LDFLAGS.

Here is a proofread diff.  I did make one substantive change: I added a comment about the test suite not running with the default maxdata and not raising memory errors with 512MB.  Is that correct?

David, I think you attached your patch to the wrong issue.

Sébastien, according to the IBM docs, the memory limit can be changed using the LDR_CNTRL variable, so running the test suite shouldn't need special linker flags, or?
(also, wouldn't it be simpler to use large pages?)

The AIX documentation says:
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=/com.ibm.itame3.doc_5.1/am51_perftune113.htm
"""It is a good idea to unset the LDR_CNTRL environment variable, so that it does not unintentionally affect other processes."""

I think people may be more interested in setting permanently the maximum memory that can be allocated by python rather than using an environment variable that may be lost and which impacts all the applications.

Another way to increase the memory available for an application is to use the ldedit command on the python executable:

ldedit -b maxdata:0x20000000 ./python

That may be easier than passing it directly to the linker.

Concerning large pages, I found that in an IBM documentation:
"""
Users are advised to be cautious in their use of the environment
variable to specify large page usage. Performance tests have shown there
can be a significant performance loss in environments where a number of
shell scripts or small, short running applications are invoked. One
example saw a shell script's execution time increase over 10 times when
the large page environment variable was specified. Customers are advised
to only set the large page environment variable around specific
applications which can benefit from large page usage.
"""
Also large pages must be activated on the system by root. This does not seem to be available by default according to the doc.

The doc would become:

"""
To allocate more segment registers to Python, you must use the linker option -bmaxdata or ldedit to specify the number of bytes you need in the data
 segment.
 
For example, if you want to allow 512MB of memory for Python (this
is enough for the test suite to run without MemroyErrors), you should
run the following command at the end of compilation:

 ldedit -b maxdata:0x20000000 ./python
"""

oops.

> I think people may be more interested in setting permanently the
> maximum memory that can be allocated by python rather than using an
> environment variable that may be lost and which impacts all the
> applications.

Agreed, but it's impossible to standardize the configure script on a
"good enough" value. Some people might need 512MB, others 2GB...

> Another way to increase the memory available for an application is to
> use the ldedit command on the python executable:
> 
> ldedit -b maxdata:0x20000000 ./python
> 
> That may be easier than passing it directly to the linker.

Sounds better. That means that it can be decoupled from the build
process.
(although for a hypothetical buildbot, I think passing LDR_CNTRL would
be the most practical solution)

I updated the documentation in issue 10709 so that it mentions ldedit rather than the linker flags (more convenient to activate support for more memory).

Once Python 3.2 is released, we may add LDR_CNTRL or ldedit in the Makefile when running buildbottest.

I will ask IBM if there is any reason not to activate support for 2GB in an application by default (impact on performances or something).
If there is none, we may systematically call "ldedit -b maxdata:0x80000000 ${srcdir}/python" when compiling python on AIX...

@David (Edelsohn) what is your take on this?

Setting the environment variable LDR_CNTRL is discouraged.  It is much better to set the value in the executable header. Best to set it at link time, but one can use ldedit.

The issue with the segments in 32 bit mode is a trade off between heap memory and shmat attached shared memory. If one wanted to use Python with some forms of shared memory, one cannot allocate all of the segments to heap. Setting maxdata too large can cause unexpected failures if shared memory is used.

Increasing the default maxdata value seems like a good idea. The default GCC for AIX builds with a larger value. But it adds the command to LDFLAGS, which is applied everywhere, but not a significant problem and cc1* do not use shared memory.

One subtlety with LDFLAG is XLC wants the flag directly (-bmaxdata:0x40000000) but GCC needs to use -Wl,-bmaxdata.