pitrou, thanks for great diagnosing program! You are right that there is no memory leak. 

We have investigated this issue a little bit more - it looks real reason for large memory consuming in the end is not fragmentation, just glibc doesn't release all free memory.

So what happens in this example (according to http://man7.org/linux/man-pages/man3/mallopt.3.html):
1) More threads leads to creating of more arenas (e.g. to more sub-heaps). It can be tested with setting environment variable MALLOC_ARENA_MAX to 1 - memory consumption in example will be reduced significantly. But this can be used in real code, performance will degrade.  
2) Heap trim threshold (M_TRIM_THRESHOLD) is calculated dynamically. Creation and deletion of small arrays leads to M_MMAP_THRESHOLD=2*4*1024*1024*sizeof(long) on 64 bit systems. So each sub-heap in each arena can have up to 64 mb (if long is 8 bytes) of untrimmed space.
It can be tested with setting environment variable MALLOC_TRIM_THRESHOLD_ to 128*1024 - memory consumption in example will be reduced significantly because dynamic calculation of trim threshold will be turned off. Again I doubt this can be used in real code.  
3) Maxiumum number of arenas vary on number of cpu's. On my system it looks like it is set to 16. So I get around 16*64 of untrimmed space.

To conclude:
There is similar bug in python tracker: https://bugs.python.org/issue11849. Message https://bugs.python.org/msg134992 states that nothing can be done in some cases (like in attached example).

One possible solution is to use jemalloc - we have tested that it doesn't have such problems. Limiting maximum number of arenas also helps, but can degrade performance.