Documentation for time.perf_counter says "does include time elapsed during sleep".

https://docs.python.org/3.8/library/time.html#time.perf_counter

But it does not work as expected in Mac OS. I learnt that perf_counter uses the underlying c function: "mach_absolute_time".

-------------------------

import time
time.get_clock_info('perf_counter')
namespace(adjustable=False, implementation='mach_absolute_time()', monotonic=True, resolution=1e-09)

-------------------------

The documentation for "mach_absolute_time" clearly states that "this clock does not increment while the system is asleep"

https://developer.apple.com/documentation/kernel/1462446-mach_absolute_time


FWIW, Mac kernel does offer another function which returns monotonic ticks "including while the system is asleep"

https://developer.apple.com/documentation/kernel/1646199-mach_continuous_time

But it seems to be available only on Mac 10.12+.

macOS 10.12 also has clock_gettime(), which would allow using the generic code path (although the current path must be kept for older versions of macOS).

perf_counter documentation says "It does include time elapsed during sleep and is system-wide." where "sleep" here means time.sleep():
https://docs.python.org/dev/library/time.html#time.perf_counter

Python clock functions don't provide any warranty regarding to system suspend or system hibernation. See PEP 418 for more details: "The behaviour of clocks after a system suspend is not defined in the documentation of new functions. The behaviour depends on the operating system: see the Monotonic Clocks section below."
https://www.python.org/dev/peps/pep-0418/#monotonic-clocks

I don't think that using mach_continuous_time() is needed, but the documentation should be clarified. The doc should explain that the behavior during system suspend is not defined (platform specific).

Wouldn’t using mach_continuous_time (and its equivalents on other platforms) wherever possible be preferable?

Or would a different API that distinguishes between clocks that track during suspend versus those that not be a better idea? Given that at least macOS and Linux offer both variants (not sure about Windows)

> Or would a different API that distinguishes between clocks that track during suspend versus those that not be a better idea? Given that at least macOS and Linux offer both variants (not sure about Windows)

I don't think that all platforms provide a clock for Python time.perf_counter() which include suspend. So I'm not sure that it's useful to only change the behavior on macOS, and only depending on the macOS version (and macOS target version).

Maybe a new clock is needed, clock which has a well defined behavior for system suspend, on any platform. Such clock may not be available on all platforms.

FYI on Python 3.3, time.monotonic() was not available on all platforms. It became available on all platforms on Python 3.5.
https://docs.python.org/dev/library/time.html#time.monotonic

> Maybe a new clock is needed, clock which has a well defined behavior for system suspend, on any platform.

I’d like to work on this, if possible. Linux and macOS support seems to be straightforward. I will have to look into other platforms.

What would be the protocol exactly? Should I float a discussion on the ideas mailing list? Open a new bpo?

I made a record of my investigations here: https://github.com/python-trio/trio/issues/1586

One reason we might care about this is that asyncio ought to be using a clock that stops ticking while suspended.

(On which note: Please don't switch macOS to use clock_gettime(CLOCK_MONOTONIC); that would be a breaking change for us!)

At least Linux, macOS, FreeBSD, and Windows all have a way to access a monotonic clock that stops ticking while the system is suspended.

> (On which note: Please don't switch macOS to use clock_gettime(CLOCK_MONOTONIC); that would be a breaking change for us!)

Right, I propose to add new clock(s) since people may rely on the current clock(s) specifications.

> At least Linux, macOS, FreeBSD, and Windows all have a way to access a monotonic clock that stops ticking while the system is suspended.

Python also tries to support AIX, OpenBSD, Solaris, Android, etc.
https://pythondev.readthedocs.io/platforms.html#best-effort-and-unofficial-platforms

I'm not sure that all "supported" platforms provide such clock. It's ok if such clock is not available on all platforms. People can fallback on monotonic/perf_counter depending on their need. For example, previously, it was common to write something like:

try: from time import monotonic
except ImportError: from time import time as monotonic # Python 2 or clock not available

@Nathaniel: I hadn't noticed that CLOCK_MONOTONIC on macOS behaves different from that clock on Linux. Sigh.

That means there's little reason to switch to CLOCK_MONOTONIC on macOS, that would just result in different behaviour between Linux and macOS. 

There is a clock with similar behaviour as the Linux clock: CLOCK_UPTIME_RAW, but switching to that instead of mach_absolute_time would just complicate the code base because we still support macOS 10.9 where clock_gettime is not available.

BTW. I'm against using mach_continuous_time, if a change is needed it should be to clock_gettime as that's the more portable API.   And given the stated goal of time.perf_counter() a change is IMHO not necessary.

> There is a clock with similar behaviour as the Linux clock: CLOCK_UPTIME_RAW

On Linux, CLOCK_UPTIME_RAW is not adjusted by NTP and it should not be used for a clock using *seconds*. NTP ensures that a clock provides seconds and is not slower or faster.

On macOS, if you want a clock which is incremented while the system is asleep, CLOCK_MONOTONIC sounds like a better choice.

I don't know if time.perf_counter() and CLOCK_MONOTONIC have similar effective resolution on macOS. time.perf_counter() should have a better resolution, but can have a worse accurary, than time.monotonic().

On macOS CLOCK_UPTIME_RAW is the same as mach_absolute_time (according to the manpage).

> On macOS CLOCK_UPTIME_RAW is the same as mach_absolute_time (according to the manpage).

https://developer.apple.com/documentation/kernel/1462446-mach_absolute_time says:

"mach_absolute_time: Returns current value of a clock that increments monotonically in tick units (starting at an arbitrary point), this clock does not increment while the system is asleep."

"Discussion: Prefer to use the equivalent clock_gettime_nsec_np(CLOCK_UPTIME_RAW) in nanoseconds."

where clock_gettime_nsec_np() is: "As a non-portable extension, the clock_gettime_nsec_np() function will return the clock value in 64-bit nanoseconds."

It doesn't say anything about NTP, whereas CLOCK_MONOTONIC_RAW seems to a clock which is not adjusted by NTP:

"""
CLOCK_MONOTONIC_RAW

clock that increments monotonically, tracking the time
since an arbitrary point like CLOCK_MONOTONIC.  How-
ever, this clock is unaffected by frequency or time
adjustments.  It should not be compared to other
system time sources.
"""

So right, it sounds like mach_absolute_time() returns *seconds*.

mach_absolute_time returns time in ticks, there's a separate API that returns the resolution of this clock (which is already used).

The manpage explicitly says that mach_absolute_time and CLOCK_UPTIME_RAW are the same clock:

     CLOCK_UPTIME_RAW   clock that increments monotonically, in the same manner as CLOCK_MONOTONIC_RAW, but that does
                        not increment while the system is asleep.  The returned value is identical to the result of
                        mach_absolute_time() after the appropriate mach_timebase conversion is applied.

Switching from mach_absolute_time to CLOCK_UPTIME_RAW would therefore bring us no improvements, and would complicate the code base because clock_gettime is only available starting from macOS 10.12.

> perf_counter documentation says "It does include time elapsed during sleep and is system-wide." where "sleep" here means time.sleep()

Apologies for misinterpreting the documentation. A clock function that includes system suspend time can be useful. Consider this as a feature request.

I noticed that on both Linux and Mac OS, the functions time.monotonic and time.perf_counter are exactly same. They do not count time elapsed during suspend.

--- CODE -----

import time
for clock in "monotonic", "perf_counter":
    print(f"{clock}: {time.get_clock_info(clock)}")

--- MAC OUTPUT ----

monotonic: namespace(adjustable=False, implementation='mach_absolute_time()', monotonic=True, resolution=1e-09)
perf_counter: namespace(adjustable=False, implementation='mach_absolute_time()', monotonic=True, resolution=1e-09)

--- LINUX OUTPUT --

monotonic: namespace(adjustable=False, implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, resolution=1e-09)
perf_counter: namespace(adjustable=False, implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, resolution=1e-09)

--------------

In Windows, perf_counter uses "QueryPerformanceCounter()" which includes "the time when the machine was in a sleep state such as standby, hibernate, or connected standby"
refer: https://docs.microsoft.com/en-us/windows/win32/sysinfo/acquiring-high-resolution-time-stamps

I learnt that "mach_continuous_time()" in Mac OS, "clock_gettime(CLOCK_BOOTTIME)" in Linux and "QueryPerformanceCounter()" in Windows all include system suspend time. It would be nice if perf_counter can be tweaked to provide similar behaviour across platforms. If it is not recommended, the mentioned functions can be exposed as a separate clock function.

Thanks.

While I agree that the behavior of perf_counter should be consistent across macOS/Linux and Windows wrt system suspend, you can already access those clocks by using time.clock_gettime [1] with appropriate clock IDs (CLOCK_MONOTONIC_RAW on macOS and CLOCK_BOOTTIME on Linux)

> While I agree that the behavior of perf_counter should be consistent across macOS/Linux and Windows wrt system suspend,

perf_counter behavior during system suspend is undefined.

[1]: https://docs.python.org/3.8/library/time.html#time.clock_gettime

[2]: https://developer.apple.com/documentation/kernel/1646199-mach_continuous_time effectively uses CLOCK_MONOTONIC_RAW on macOS.

> perf_counter behavior during system suspend is undefined

Does the same apply for time.monotonic? I would argue that the difference in behavior between Linux/macOS and Windows is unreasonable; given that time.monotonic exists for measuring time intervals (which are not necessarily required to be of short duration – unlike perf_counter).

> Does the same apply for time.monotonic? I would argue that the 
> difference in behavior between Linux/macOS and Windows is 
> unreasonable; given that time.monotonic exists for measuring time 
> intervals

For time.monotonic in Windows, Python could switch to using QueryUnbiasedInterruptTime [1], which excludes periods in which the system is suspended or hibernated. In Windows 10, QueryUnbiasedInterruptTimePrecise [2] can be used, which directly reads the counter in the hardware timer instead of the value at the last timer interrupt.

For time.time in Windows, Python 3.10 should switch to using GetSystemTimePreciseAsFileTime [3] instead of GetSystemTimeAsFileTime. 

For time.perf_counter in Windows, QueryPerformanceCounter remains the best option since it uses the invariant TSC in the CPU if available.

---

Note that, unlike GetTickCount64, the resolution for QueryUnbiasedInterruptTime isn't lpTimeIncrement from GetSystemTimeAdjustment [4]. lpTimeIncrement is the initial default and upper bound for the interrupt period, which can be adjusted down with timeBeginPeriod or the undocumented system call NtSetTimerResolution. The former can lower the interrupt period from the default of about 10-16 ms (usually 15.625 ms, i.e. 64 interrupts/second) down to about 1 ms, while NtSetTimerResolution goes down to about 0.5 ms. It's not free, however, else it would be the default setting. More frequent timer interrupts can decrease system performance and increase power consumption.

---

[1] https://docs.microsoft.com/en-us/windows/win32/api/realtimeapiset/nf-realtimeapiset-queryunbiasedinterrupttime

[2] https://docs.microsoft.com/en-us/windows/win32/api/realtimeapiset/nf-realtimeapiset-queryunbiasedinterrupttimeprecise

[3] https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getsystemtimepreciseasfiletime

[4] https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getsystemtimeadjustment

> For time.time in Windows, Python 3.10 should switch to using GetSystemTimePreciseAsFileTime [3] instead of GetSystemTimeAsFileTime. 

It's already tracked by bpo-19007: "precise time.time() under Windows 8: use GetSystemTimePreciseAsFileTime".