Author vstinner
Recipients serhiy.storchaka, vstinner, yselivanov
Date 2016-01-28.15:52:43
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1453996363.57.0.359000785259.issue26233@psf.upfronthosting.co.za>
In-reply-to
Content
I dislike allocating large memory block on the stack, we should not abuse it.

I propose instead to store the memory block allocated on the heap in the epoll object. Attached patch implements this idea.

The buffer is stolen from the epoll object when epoll.poll() is called. Then it is stored again inside the epoll object, except if another thread stored a larger buffer in the meanwhile. The largest buffer is also kept.

The buffer exchange is protected by the GIL.

My patch also overallocates the buffer by 50% to avoid calling realloc() to many times. Using the selectors module, maxevents is the number of registered FD. Using asyncio, the number of registered FD changes a lot.

Side effect: the memory block is not released after the call to epoll.poll(). If you can it with an insane max_events, it will bloat your memory until your close the poller.

If the approach is considered interested and efficient, I can work on a similar patch for other pollers.
History
Date User Action Args
2016-01-28 15:52:43vstinnersetrecipients: + vstinner, serhiy.storchaka, yselivanov
2016-01-28 15:52:43vstinnersetmessageid: <1453996363.57.0.359000785259.issue26233@psf.upfronthosting.co.za>
2016-01-28 15:52:43vstinnerlinkissue26233 messages
2016-01-28 15:52:43vstinnercreate