Created on 2006-06-19 19:53 by qopit, last changed 2014-07-29 20:58 by haypo. This issue is now closed.
|msg28841 - (view)||Author: Russell Warren (qopit)||Date: 2006-06-19 19:53|
THE ISSUE... --- threading.py imports time.time as _time. On win32 systems, time.time() periodically reads the system time to figure out when to fire an Event. System time can change while waiting for an Event! eg: If the system time is changed while a threading.Timer is pending, the execution time is affected by the time change. eg: set a pending Timer and then change the clock back an hour - this causes your Timer to fire an hour later. This is clearly not desirable. A FIX... --- A fix for this is to use time.clock() on win32 systems instead. Once I found the problem, I currently just fix it by overriding threading._time to be time.clock. Right now I do this in every file that uses threading.Timer. COMMENTS... --- The penalty for this is that there will be a rollover problem eventaully... when the 64-bit performance counter rolls over in 30+ years of continuous pc operation. I'd much rather have this near impossible event than the very likely system time change. This is a general problem I find with the time module and I often have to switch between time() and clock() dependent on operating system, but I only work with win32 and Linux. The issue is that if you want a high resolution and extended rollover counter, it is a different call on each system.
|msg28842 - (view)||Author: Russell Warren (qopit)||Date: 2006-06-26 21:59|
Logged In: YES user_id=1542586 This is an issue for anything that uses threading.py's _time function. This includes _Condition.wait(), which screws up both Conditions and Events, which then screw up (for example) the core Queue.Queue timeout implementation. threading.Thread.join also uses the _time call, so it could also get screwed by changes to the local time on Win32.
|msg28843 - (view)||Author: Jim Jewett (jimjjewett)||Date: 2006-06-27 15:04|
Logged In: YES user_id=764593 Logically, these calls should always use clock, since they don't care about the actual time; they only want a baseline for computing elapsed time. Are you saying that they should still use clock-time on linux anyway, because of resolution issues?
|msg28844 - (view)||Author: Russell Warren (qopit)||Date: 2006-06-27 20:01|
Logged In: YES user_id=1542586 No - just stating that clock() is definitely a better solution for win32. As you say, any system should just use a time-indpendent uptime counter that is as high resolution as possible. I don't know how to get this on linux, but I do seem to recall that, on linux, time() is higher resolution than clock() for some reason. If linux has no performance counter equivalent (isn't it a hardware thing anyway?) I have no clue which is worse... low resolution, or local time change issues. The first is limiting all the time, the second results in wacky and sporadic errors that people might not expect.
|msg28845 - (view)||Author: Joseph Tate (jtate)||Date: 2006-10-09 13:59|
Logged In: YES user_id=55276 This is also currently a problem under Linux when the clock is set back.
|msg28846 - (view)||Author: Joseph Tate (jtate)||Date: 2007-03-28 15:07|
On Linux time.clock() is CPU time. And has no bearing at all on real time. On windows time.clock() is real time since CPU time is not available from the operating system. What if instead of replacing time.time() with time.clock(), the Queue and Event wait loops just kicked out if "remaining time" was larger than the initial wait time? Let the outer loop continue, and the next time wait is called, the "remaining time" will be calculated correctly. Otherwise, we could compare "remaining time" between one loop iteration and the subsequent and then *approximate* how much the time has changed to adjust the "end time" appropriately.
|msg28847 - (view)||Author: xiaowen (xiaowen)||Date: 2007-03-29 01:41|
Patch implementing what jtate described: --- python/Lib/threading.py 2007-03-29 09:13:53.000000000 +0800 +++ python/Lib/threading.py.new 2007-03-29 09:26:44.000000000 +0800 @@ -222,15 +222,19 @@ # than 20 times per second (or the timeout time remaining). endtime = _time() + timeout delay = 0.0005 # 500 us -> initial delay of 1 ms + remaining = max(timeout, 0) while True: gotit = waiter.acquire(0) if gotit: break - remaining = endtime - _time() if remaining <= 0: break delay = min(delay * 2, remaining, .05) _sleep(delay) + if endtime - _time() > remaining: + # Time must have been moved back, so readjust endtime. + endtime = _time() + remaining - delay + remaining = endtime - _time() if not gotit: if __debug__: self._note("%s.wait(%s): timed out", self, timeout) --- python/Lib/Queue.py 2007-01-01 00:38:56.000000000 +0800 +++ python/Lib/Queue.py.new 2007-01-01 00:42:14.000000000 +0800 @@ -127,7 +127,7 @@ endtime = _time() + timeout while self._full(): remaining = endtime - _time() - if remaining <= 0.0: + if remaining <= 0.0 or remaining > timeout: raise Full self.not_full.wait(remaining) self._put(item) @@ -169,7 +169,7 @@ endtime = _time() + timeout while self._empty(): remaining = endtime - _time() - if remaining <= 0.0: + if remaining <= 0.0 or remaining > timeout: raise Empty self.not_empty.wait(remaining) item = self._get()
|msg113754 - (view)||Author: Antoine Pitrou (pitrou) *||Date: 2010-08-13 10:30|
The wait loop in Condition.wait() has disappeared under 3.2 (we use builtin lock-with-timeout primitives instead). Overall, I don't think this issue is very important. Changing the system time *backwards* can cause all kinds of issues in daemons and other long-running programs, which generally assume a monotonous time function. Also, it could be disputed what the preferred semantics are (observe "real" time or observe the computer clock), and the answer is probably application-specific.
|msg160936 - (view)||Author: STINNER Victor (haypo) *||Date: 2012-05-16 22:38|
|msg224248 - (view)||Author: Mark Lawrence (BreamoreBoy) *||Date: 2014-07-29 20:46|
PEP 418 states "time.monotonic(): timeout and scheduling, not affected by system clock updates" and has also deprecated time.clock() so I believe this can be closed as "out of date".
|msg224249 - (view)||Author: STINNER Victor (haypo) *||Date: 2014-07-29 20:58|
As Antoine wrote, Condition.wait() was rewritten in Python 3.2 to implement timeout using the native OS "acquire a lock with a timeout" function. So the initial concern is already fixed. This change is huge, we are not going to backport new lock timeouts in Python 2.7, it's too risky. It's time to upgrade to Python 3! There is still a *corner case* when the function is interrupted by a signal, we use the system clock to recompute the new timeout. This corner case is addresses by the issue #22043.
|2014-07-29 20:58:03||haypo||set||status: open -> closed|
messages: + msg224249
messages: + msg224248
|2012-05-16 22:38:05||haypo||set||messages: + msg160936|
messages: + msg113754
jimjjewett, qopit, jtate, xiaowen|
components: + Windows
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 3.0
stage: test needed
versions: + Python 2.6, Python 3.0