This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author pitrou
Recipients dmalcolm, eli.bendersky, flox, kaifeng, neologix, pitrou, python-dev
Date 2011-11-25.22:00:00
SpamBayes Score 1.6975e-08
Marked as misclassified No
Message-id <1322258101.3272.41.camel@localhost.localdomain>
In-reply-to <>
> On my box:
> default:
> $ ./python -m timeit -s "n=300000; f=open('/tmp/10MB.bin', 'rb');
> b=bytearray(n)" ";f.readinto(b)"
> 1000 loops, best of 3: 640 usec per loop
> default without patch ("$ hg revert -r 68258 Objects/obmalloc.c && make"):
> $ ./python -m timeit -s "n=300000; f=open('/tmp/10MB.bin', 'rb');
> b=bytearray(n)" ";f.readinto(b)"
> 1000 loops, best of 3: 663 usec per loop
> I'm just observing a random variance (but my computer is maybe too
> slow to notice).

Hmm, quite slow indeed, are you sure you're not running in debug mode?

> However, I really don't see how the patch could play a role here.
> Concerning the slight performance regression, if it's a problem, I see
> two options:
> - revert the patch
> - replace calls to malloc()/free() by mmap()/munmap() to allocate/free
> arenas (but I'm not sure anonymous mappings are supported by every OS
> out there, so this might lead to some ugly #ifdef's...)

If the performance regression is limited to read(), I don't think it's
really an issue, but using mmap/munmap explicitly would probably benicer
anyway (1° because it lets the glibc choose whatever heuristic is best,
2° because it would help release memory on more systems than just glibc
systems). I think limiting ourselves to systems which have
MMAP_ANONYMOUS is good enough.

Here is what the glibc malloc does btw:

   Nearly all versions of mmap support MAP_ANONYMOUS,
   so the following is unlikely to be needed, but is
   supplied just in case.


static int dev_zero_fd = -1; /* Cached file descriptor for /dev/zero. */

#define MMAP(addr, size, prot, flags) ((dev_zero_fd < 0) ? \
 (dev_zero_fd = open("/dev/zero", O_RDWR), \
  mmap((addr), (size), (prot), (flags), dev_zero_fd, 0)) : \
   mmap((addr), (size), (prot), (flags), dev_zero_fd, 0))


#define MMAP(addr, size, prot, flags) \
 (mmap((addr), (size), (prot), (flags)|MAP_ANONYMOUS, -1, 0))

Date User Action Args
2011-11-25 22:00:01pitrousetrecipients: + pitrou, eli.bendersky, flox, dmalcolm, neologix, python-dev, kaifeng
2011-11-25 22:00:00pitroulinkissue11849 messages
2011-11-25 22:00:00pitroucreate