Author rhettinger
Recipients JohanAR, davin, gvanrossum, itamarst, ncoghlan, pitrou, python-dev, rhettinger, sbt, serhiy.storchaka, tim.peters, yselivanov, zzzeek
Date 2017-09-05.06:19:52
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1504592393.59.0.0585305432289.issue14976@psf.upfronthosting.co.za>
In-reply-to
Content
> I can't think of a reason why deque would need to release the GIL.

Yes, in deque.append(item) and deque.popleft() are atomic.  Likewise list.append() and list.pop() are atomic.  The heappush() and heappop() operations aren't as fortunate since they call __lt__() on arbitrary Python objects.

> perhaps we should indeed follow Antoine's advice and implement
> a different queue that has fewer features but is guaranteed 
> to be usable by signal handlers and GC callbacks 
> (including __del__).

Is the idea to use a regular non-reentrant lock but write the whole thing in C to avoid running any pure python instructions (any of which could allow a signal handler to run)?

If so, it seems like the only feature that needs to be dropped is subclassing.  The maxsize feature and unfinished tasks tracking could still be supported.  Also, I suppose the guarantees would have be marked as CPython implementation details, leaving the other implementations to fend for themselves.

Or were you thinking about a pure python simplified queue class?  If so, an RLock() will likely be required (the act of assigning deque.popleft's return value to a pure python variable can allow a pending signal to be handled before the lock could be released).

One other thought:  Would it make sense get() and put() to add gc.disable() and gc.enable() whenever GC is already enabled?  That would eliminate a source of reentrancy.
History
Date User Action Args
2017-09-05 06:19:53rhettingersetrecipients: + rhettinger, gvanrossum, tim.peters, ncoghlan, pitrou, zzzeek, python-dev, sbt, serhiy.storchaka, JohanAR, yselivanov, itamarst, davin
2017-09-05 06:19:53rhettingersetmessageid: <1504592393.59.0.0585305432289.issue14976@psf.upfronthosting.co.za>
2017-09-05 06:19:53rhettingerlinkissue14976 messages
2017-09-05 06:19:52rhettingercreate