Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sched doesn't handle at the expected time those events which are added after scheduler starts #64325

Closed
Julian mannequin opened this issue Jan 5, 2014 · 13 comments
Closed
Labels
3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@Julian
Copy link
Mannequin

Julian mannequin commented Jan 5, 2014

BPO 20126
Nosy @giampaolo, @bitdancer, @serhiy-storchaka, @MojoVampire, @iritkatriel

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-01-05.03:00:23.335>
labels = ['type-bug', 'library', '3.11']
title = "sched doesn't handle at the expected time those events which are added after scheduler starts"
updated_at = <Date 2021-11-27.08:59:08.365>
user = 'https://bugs.python.org/julian'

bugs.python.org fields:

activity = <Date 2021-11-27.08:59:08.365>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2014-01-05.03:00:23.335>
creator = 'julian'
dependencies = []
files = []
hgrepos = []
issue_num = 20126
keywords = []
message_count = 11.0
messages = ['207336', '207343', '207391', '207405', '207406', '207407', '207408', '207543', '244345', '360285', '407107']
nosy_count = 7.0
nosy_names = ['giampaolo.rodola', 'r.david.murray', 'serhiy.storchaka', 'julian', 'rgov', 'josh.r', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20126'
versions = ['Python 3.11']

@Julian
Copy link
Mannequin Author

Julian mannequin commented Jan 5, 2014

Events added after the scheduler is running, with a delay value before the next event, do not run at the correct time.

import sched
import time
import threading

def event( eventNum ):
	print( 'event', eventNum, time.time() )

s = sched.scheduler()
s.enter( 0,1,event, (0,) )
s.enter(10,1,event, (1,) )
t = threading.Thread( target = s.run )
t.start()
s.enter( 5,1,event, (2,) )

OUTPUT
event 0 1388890197.7716181
event 2 1388890207.7340584
event 1 1388890207.7347224

@Julian Julian mannequin added the type-bug An unexpected behavior, bug, or error label Jan 5, 2014
@bitdancer
Copy link
Member

I believe this is a duplicate of bpo-16165, which has already been fixed.

@Julian
Copy link
Mannequin Author

Julian mannequin commented Jan 5, 2014

i saw that issue before i submitted this one, but it says it was fixed a year ago. i assumed a patch from a year ago would be in 3.3.1.

when i see an issue closed with a patch accepted, how do i determine which python version will contain that patch?

@bitdancer
Copy link
Member

Search for the issue number in Misc/NEWS. It looks like it was indeed included in 3.3.1. Can you figure out why that fix didn't fix your problem?

@bitdancer bitdancer reopened this Jan 5, 2014
@serhiy-storchaka
Copy link
Member

This is because the scheduler already has executed first event and slept on delaying second event before you have added third event. This is unavoidable for this implementation. sched.scheduler can't be waked when sleep.

@Julian
Copy link
Mannequin Author

Julian mannequin commented Jan 5, 2014

i looks like a different issue. 16165 says you can't add events while the scheduler is running. in my case, i can add them, but they don't run at the correct time.

i'll guess at what the problem is... when you start the scheduler, the thread sleeps until the first scheduled event. when you add an event after the scheduler is running, the scheduler doesn't re-evaluate how long it should sleep. sched.enter(), sched.enterabs(), and sched.cancel() should probably send a signal to the scheduler thread to recalculate the time of the next event.

@Julian
Copy link
Mannequin Author

Julian mannequin commented Jan 5, 2014

ok, makes sense. can we put a note in the manual about this limitation?

@giampaolo
Copy link
Contributor

I don't think this should be documented as personally I wouldn't expect this use case to be working in the first place.

@MojoVampire
Copy link
Mannequin

MojoVampire mannequin commented May 28, 2015

Why would you not expect this to work in the first place? I'd think a relatively simple use case for schedulers is using a single thread to do the work of a Timer instead of spawning a thread for every Timer. In that sort of use case, the "timer worker" is constantly pulling tasks and other thread(s) are adding them; for an empty event queue, if thread A adds a task 'a' that delays for 10 minutes and thread B adds a task 'b' that delays for 1 minute, the time at which task 'b' executes changes dramatically based on which thread squeaks out a win on the race to insert into the scheduler queue. As long as task 'a' wins the race, a million other tasks could be scheduled to run before it, but all of them will be stuck behind task 'a' no matter their priority or delay.

It should be possible to fix this without too much trouble (and without polling loops), but it would require scheduler to remove the option to customize delayfunc; you'd change the scheduler's RLock to a Condition, have mutations to the event queue notify the Condition, and the run method would replace sleeping outside the locked scope with a wait w/timeout inside it.

The only thing blocking a fix for this is backwards compatibility. If people are okay with breaking that, I'll happily contribute a patch, but I want some indication of whether back compat can be sacrificed to make sched actually useful in threaded contexts.

@rgov
Copy link
Mannequin

rgov mannequin commented Jan 20, 2020

This absolutely should be documented. If adding an earlier event is not supported then it should raise an exception. Appearing the enqueue the event but not triggering the callback is especially confusing. It may be obvious behavior to someone who has spent time developing this module or working with it but not to someone who just wants to, e.g., build an alarm clock or calendar app.

@iritkatriel
Copy link
Member

Reproduced on 3.11.

@iritkatriel iritkatriel added 3.11 only security fixes stdlib Python modules in the Lib dir labels Nov 26, 2021
@iritkatriel iritkatriel changed the title sched doesn't handle events added after scheduler starts sched doesn't handle events added after scheduler starts as expected Nov 26, 2021
@iritkatriel iritkatriel changed the title sched doesn't handle events added after scheduler starts as expected sched doesn't handle at the expected time those events which are added after scheduler starts Nov 27, 2021
@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@semiversus
Copy link

Just for reference I found a working solution: https://github.com/phluentmed/event-scheduler

Dealing with events entering before the actual task is ending gets far more complicated than the current sched.scheduler design. The referenced solution uses an additional thread and threading.Condition to handle that case. It is not compatible with the sched.scheduler interface.

@larryhastings
Copy link
Contributor

This is not fixable without breaking the sched.scheduler interface. We might add a new scheduler object in the future, but this bug won't be fixed on the existing one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

6 participants