Author gvanrossum
Recipients Brunsgaard, gvanrossum, vstinner, yselivanov
Date 2016-02-01.18:38:50
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAP7+vJJ4ktpBOhi8OFEwmSbUQYfYtzwinu99UWOefL_3UorUsA@mail.gmail.com>
In-reply-to <1454347680.42.0.147949880695.issue26259@psf.upfronthosting.co.za>
Content
> The easy solution was to check if the queue was empty and skip the read (get call) if there was nothing in the queue.

There's an API for that -- get_nowait().

I'm happy to review a fix if you come up with one. But I currently
don't have time to devise a fix myself. (Not even a docfix, if you
think that's sufficient, but if you submit one it will swiftly be
applied.)

On Mon, Feb 1, 2016 at 9:28 AM, Jonas Brunsgaard <report@bugs.python.org> wrote:
>
> Jonas Brunsgaard added the comment:
>
> In my particular case, I developed an application close to beanstalkd, but with redis as "engine". I did create a callbackback reader class for users to subclass, the callbackreader is checking every second, on every tube(queue.Object). If new data has arrived for processing (this subroutine is using queue.get with wait_for). Maybe asyncio.Condition would have been the better choice. The easy solution was to check if the queue was empty and skip the read (get call) if there was nothing in the queue.
>
> Before my fix, over a week the program would take up 10 Gigs of memory in our staging environment if nothing was touched, so I was assigned to investigate the cause. I think the current behavior is undesirable and cumbersome to see through, and if not changed there should be some kind of note in the documentation, so other good python folks will have a better chance to understand the behavior without reading the cpython asyncio queue implementation.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue26259>
> _______________________________________
History
Date User Action Args
2016-02-01 18:38:50gvanrossumsetrecipients: + gvanrossum, vstinner, yselivanov, Brunsgaard
2016-02-01 18:38:50gvanrossumlinkissue26259 messages
2016-02-01 18:38:50gvanrossumcreate