Title: threading.Timer/timeouts break on change of win32 local time
Type: behavior Stage: test needed
Components: Library (Lib), Windows Versions: Python 3.1, Python 3.2, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, haypo, jimjjewett, jtate, pitrou, qopit, xiaowen
Priority: normal Keywords: patch

Created on 2006-06-19 19:53 by qopit, last changed 2014-07-29 20:58 by haypo. This issue is now closed.

Messages (11)
msg28841 - (view) Author: Russell Warren (qopit) Date: 2006-06-19 19:53
--- imports time.time as _time.

On win32 systems, time.time() periodically reads the
system time to figure out when to fire an Event.

System time can change while waiting for an Event!

eg:  If the system time is changed while a
threading.Timer is pending, the execution time is
affected by the time change.  eg: set a pending Timer
and then change the clock back an hour - this causes
your Timer to fire an hour later.  This is clearly not

A FIX...
A fix for this is to use time.clock() on win32 systems
instead.  Once I found the problem, I currently just
fix it by overriding threading._time to be time.clock.
 Right now I do this in every file that uses

The penalty for this is that there will be a rollover
problem eventaully... when the 64-bit performance
counter rolls over in 30+ years of continuous pc
operation.  I'd much rather have this near impossible
event than the very likely system time change.

This is a general problem I find with the time module
and I often have to switch between time() and clock()
dependent on operating system, but I only work with
win32 and Linux.  The issue is that if you want a high
resolution and extended rollover counter, it is a
different call on each system.

msg28842 - (view) Author: Russell Warren (qopit) Date: 2006-06-26 21:59
Logged In: YES 

This is an issue for anything that uses's _time

This includes _Condition.wait(), which screws up both
Conditions and Events, which then screw up (for example) the
core Queue.Queue timeout implementation.

threading.Thread.join also uses the _time call, so it could
also get screwed by changes to the local time on Win32.

msg28843 - (view) Author: Jim Jewett (jimjjewett) Date: 2006-06-27 15:04
Logged In: YES 

Logically, these calls should always use clock, since they 
don't care about the actual time; they only want a baseline 
for computing elapsed time.

Are you saying that they should still use clock-time on 
linux anyway, because of resolution issues?
msg28844 - (view) Author: Russell Warren (qopit) Date: 2006-06-27 20:01
Logged In: YES 

No - just stating that clock() is definitely a better
solution for win32.

As you say, any system should just use a time-indpendent
uptime counter that is as high resolution as possible.  I
don't know how to get this on linux, but I do seem to recall
that, on linux, time() is higher resolution than clock() for
some reason.

If linux has no performance counter equivalent (isn't it a
hardware thing anyway?) I have no clue which is worse... low
resolution, or local time change issues.  The first is
limiting all the time, the second results in wacky and
sporadic errors that people might not expect.
msg28845 - (view) Author: Joseph Tate (jtate) Date: 2006-10-09 13:59
Logged In: YES 

This is also currently a problem under Linux when the clock
is set back.
msg28846 - (view) Author: Joseph Tate (jtate) Date: 2007-03-28 15:07
On Linux time.clock() is CPU time.  And has no bearing at all on real time.  On windows time.clock() is real time since CPU time is not available from the operating system.

What if instead of replacing time.time() with time.clock(), the Queue and Event wait loops just kicked out if "remaining time" was larger than the initial wait time?  Let the outer loop continue, and the next time wait is called, the "remaining time" will be calculated correctly.

Otherwise, we could compare "remaining time" between one loop iteration and the subsequent and then *approximate* how much the time has changed to adjust the "end time" appropriately.
msg28847 - (view) Author: xiaowen (xiaowen) Date: 2007-03-29 01:41
Patch implementing what jtate described:

--- python/Lib/     2007-03-29 09:13:53.000000000 +0800
+++ python/Lib/ 2007-03-29 09:26:44.000000000 +0800
@@ -222,15 +222,19 @@
                 # than 20 times per second (or the timeout time remaining).
                 endtime = _time() + timeout
                 delay = 0.0005 # 500 us -> initial delay of 1 ms
+                remaining = max(timeout, 0)
                 while True:
                     gotit = waiter.acquire(0)
                     if gotit:
-                    remaining = endtime - _time()
                     if remaining <= 0:
                     delay = min(delay * 2, remaining, .05)
+                    if endtime - _time() > remaining:
+                        # Time must have been moved back, so readjust endtime.
+                        endtime = _time() + remaining - delay
+                    remaining = endtime - _time()
                 if not gotit:
                     if __debug__:
                         self._note("%s.wait(%s): timed out", self, timeout)
--- python/Lib/         2007-01-01 00:38:56.000000000 +0800
+++ python/Lib/     2007-01-01 00:42:14.000000000 +0800
@@ -127,7 +127,7 @@
                 endtime = _time() + timeout
                 while self._full():
                     remaining = endtime - _time()
-                    if remaining <= 0.0:
+                    if remaining <= 0.0 or remaining > timeout:
                         raise Full
@@ -169,7 +169,7 @@
                 endtime = _time() + timeout
                 while self._empty():
                     remaining = endtime - _time()
-                    if remaining <= 0.0:
+                    if remaining <= 0.0 or remaining > timeout:
                         raise Empty
             item = self._get()
msg113754 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-13 10:30
The wait loop in Condition.wait() has disappeared under 3.2 (we use builtin lock-with-timeout primitives instead).

Overall, I don't think this issue is very important. Changing the system time *backwards* can cause all kinds of issues in daemons and other long-running programs, which generally assume a monotonous time function. Also, it could be disputed what the preferred semantics are (observe "real" time or observe the computer clock), and the answer is probably application-specific.
msg160936 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2012-05-16 22:38
> imports time.time as _time.

threading is now using a monotonic clock: see the issue #14222 and the PEP 418.
msg224248 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-29 20:46
PEP 418 states "time.monotonic(): timeout and scheduling, not affected by system clock updates" and has also deprecated time.clock() so I believe this can be closed as "out of date".
msg224249 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-07-29 20:58
As Antoine wrote, Condition.wait() was rewritten in Python 3.2 to implement timeout using the native OS "acquire a lock with a timeout" function. So the initial concern is already fixed. This change is huge, we are not going to backport new lock timeouts in Python 2.7, it's too risky. It's time to upgrade to Python 3!

There is still a *corner case* when the function is interrupted by a signal, we use the system clock to recompute the new timeout. This corner case is addresses by the issue #22043.
Date User Action Args
2014-07-29 20:58:03hayposetstatus: open -> closed
resolution: fixed
messages: + msg224249
2014-07-29 20:46:54BreamoreBoysetnosy: + BreamoreBoy
messages: + msg224248
2012-05-16 22:38:05hayposetmessages: + msg160936
2012-05-16 16:16:57pitrousetnosy: + haypo
2010-08-13 10:30:44pitrousetnosy: + pitrou
messages: + msg113754
2010-08-13 04:12:42brian.curtinsetnosy: jimjjewett, qopit, jtate, xiaowen
components: + Windows
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 3.0
2009-03-30 06:32:59ajaksu2setkeywords: + patch
stage: test needed
type: behavior
versions: + Python 2.6, Python 3.0
2006-06-19 19:53:59qopitcreate