classification
Title: sched doesn't handle events added after scheduler starts
Type: behavior Stage:
Components: Versions: Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, josh.r, julian, r.david.murray, serhiy.storchaka
Priority: normal Keywords:

Created on 2014-01-05 03:00 by julian, last changed 2015-05-28 20:52 by josh.r.

Messages (9)
msg207336 - (view) Author: julian (julian) Date: 2014-01-05 03:00
Events added after the scheduler is running, with a delay value before the next event, do not run at the correct time.

import sched
import time
import threading

def event( eventNum ):
	print( 'event', eventNum, time.time() )

s = sched.scheduler()
s.enter( 0,1,event, (0,) )
s.enter(10,1,event, (1,) )
t = threading.Thread( target = s.run )
t.start()
s.enter( 5,1,event, (2,) )


OUTPUT
event 0 1388890197.7716181
event 2 1388890207.7340584
event 1 1388890207.7347224
msg207343 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-05 03:40
I believe this is a duplicate of issue 16165, which has already been fixed.
msg207391 - (view) Author: julian (julian) Date: 2014-01-05 19:04
i saw that issue before i submitted this one, but it says it was fixed a year ago. i assumed a patch from a year ago would be in 3.3.1.

when i see an issue closed with a patch accepted, how do i determine which python version will contain that patch?
msg207405 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-05 22:27
Search for the issue number in Misc/NEWS.  It looks like it was indeed included in 3.3.1.  Can you figure out why that fix didn't fix your problem?
msg207406 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-05 23:00
This is because the scheduler already has executed first event and slept on delaying second event before you have added third event. This is unavoidable for this implementation.  sched.scheduler can't be waked when sleep.
msg207407 - (view) Author: julian (julian) Date: 2014-01-05 23:05
i looks like a different issue. 16165 says you can't add events while the scheduler is running. in my case, i can add them, but they don't run at the correct time.

i'll guess at what the problem is... when you start the scheduler, the thread sleeps until the first scheduled event. when you add an event after the scheduler is running, the scheduler doesn't re-evaluate how long it should sleep. sched.enter(), sched.enterabs(), and sched.cancel() should probably send a signal to the scheduler thread to recalculate the time of the next event.
msg207408 - (view) Author: julian (julian) Date: 2014-01-05 23:20
ok, makes sense. can we put a note in the manual about this limitation?
msg207543 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2014-01-07 14:54
I don't think this should be documented as personally I wouldn't expect this use case to be working in the first place.
msg244345 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2015-05-28 20:52
Why would you not expect this to work in the first place? I'd think a relatively simple use case for schedulers is using a single thread to do the work of a Timer instead of spawning a thread for every Timer. In that sort of use case, the "timer worker" is constantly pulling tasks and other thread(s) are adding them; for an empty event queue, if thread A adds a task 'a' that delays for 10 minutes and thread B adds a task 'b' that delays for 1 minute, the time at which task 'b' executes changes dramatically based on which thread squeaks out a win on the race to insert into the scheduler queue. As long as task 'a' wins the race, a million other tasks could be scheduled to run before it, but all of them will be stuck behind task 'a' no matter their priority or delay.

It should be possible to fix this without too much trouble (and without polling loops), but it would require scheduler to remove the option to customize delayfunc; you'd change the scheduler's RLock to a Condition, have mutations to the event queue notify the Condition, and the run method would replace sleeping outside the locked scope with a wait w/timeout inside it.

The only thing blocking a fix for this is backwards compatibility. If people are okay with breaking that, I'll happily contribute a patch, but I want some indication of whether back compat can be sacrificed to make sched actually useful in threaded contexts.
History
Date User Action Args
2015-05-28 20:52:51josh.rsetnosy: + josh.r
messages: + msg244345
2014-01-07 14:54:43giampaolo.rodolasetnosy: + giampaolo.rodola
messages: + msg207543
2014-01-05 23:20:05juliansetmessages: + msg207408
2014-01-05 23:05:26juliansetmessages: + msg207407
2014-01-05 23:00:33serhiy.storchakasetmessages: + msg207406
2014-01-05 22:28:47r.david.murraysetnosy: + serhiy.storchaka
2014-01-05 22:27:45r.david.murraysetstatus: closed -> open
superseder: sched.scheduler.run() blocks scheduler ->
messages: + msg207405

resolution: duplicate ->
stage: resolved ->
2014-01-05 19:04:58juliansetmessages: + msg207391
2014-01-05 03:40:07r.david.murraysetstatus: open -> closed

superseder: sched.scheduler.run() blocks scheduler

nosy: + r.david.murray
messages: + msg207343
resolution: duplicate
stage: resolved
2014-01-05 03:00:23juliancreate