classification
Title: time.thread_time isn't outputting in nanoseconds in AIX
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, belopolsky, p-ganssle, pitrou, vstinner
Priority: normal Keywords: patch

Created on 2020-04-05 01:15 by BTaskaya, last changed 2020-05-16 10:14 by BTaskaya. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19381 merged BTaskaya, 2020-04-05 01:19
Messages (9)
msg365807 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python triager) Date: 2020-04-05 01:15
The resolution for thread_time is really low on AIX, but fortunately there is a way to get thread time in nanoseconds with thread_cputime.

-bash-4.4$ ./python
Python 3.9.0a5+ (heads/master:909f4a3, Apr  4 2020, 20:15:24) [C] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.thread_time()
0.02
msg368921 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-15 03:01
Which implementation is currently used on AIX? What's the output of the following command?

$ ./python
Python 3.9.0a6+ (heads/master:4a12d12186, May 15 2020, 04:55:17) 
>>> import time; time.get_clock_info('thread_time')
namespace(adjustable=False, implementation='clock_gettime(CLOCK_THREAD_CPUTIME_ID)', monotonic=True, resolution=1e-09)
msg368945 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python triager) Date: 2020-05-15 14:02
current:
>>> import time
>>> import time
>>> time.get_clock_info('thread_time')
namespace(adjustable=False, implementation='clock_gettime(CLOCK_THREAD_CPUTIME_ID)', monotonic=True, resolution=0.01)
>>> time.thread_time()
0.07

PR 19381:
>>> import time
>>> time.get_clock_info('thread_time')
namespace(adjustable=False, implementation='thread_cputime()', monotonic=True, resolution=1e-09)
>>> time.thread_time()
0.002379953
msg368946 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-15 14:20
I found this documentation on AIX thread_cputime():
https://www.ibm.com/support/knowledgecenter/ssw_aix_71/t_bostechref/thread_cputime.html

"""
Syntax

#include <sys/thread.h>
int thread_cputime (tid, ctime)
tid_t tid;
thread_cputime_t * ctime ;
typedef struct {
    uint64_t utime; /* User time in nanosenconds */
    uint64_t stime; /* System time in nanoseconds */
} thread_cputime_t;

Description

The thread_cputime subroutine allows a thread to query the CPU usage of the specified thread (tid) in the same process or in another process. If a value of -1 is passed in the tid parameter field, then the CPU usage of the calling thread is retrieved.

CPU usage is not the same as the total life of the thread in real time, rather it is the actual amount of CPU time consumed by the thread since it was created. The CPU usage retrieved by this subroutine contains the CPU time consumed by the requested thread tid in user space (utime) and system space (stime).

The thread to be queried is identified using the kernel thread ID which has global scope. This can be obtained by the application using the thread_self system call. Only 1:1 thread mode is supported. The result for M:N thread mode is undefined.

The CPU usage of a thread that is not the calling thread will be current as of the last time the thread was dispatched. This value will be off by a small amount if the target thread is currently running.
"""

Ok good, it returns the user time *and* the system time, and it's the thread CPU time.

So it sounds reasonable to use it to implement time.thread_time().

By the way, the v8 project calls thread_cputime() on AIX when clock_gettime(CLOCK_THREAD_CPUTIME_ID) is requested, also to get better resolution:
https://github.com/v8/v8/blob/a5038c42283a09f65c44229907123e15a779feb7/src/base/platform/time.cc#L68

// On AIX clock_gettime for CLOCK_THREAD_CPUTIME_ID outputs time with
// resolution of 10ms. thread_cputime API provides the time in ns
#if defined(V8_OS_AIX)
  thread_cputime_t tc;
  if (clk_id == CLOCK_THREAD_CPUTIME_ID) {
#if defined(__PASE__)  // CLOCK_THREAD_CPUTIME_ID clock not supported on IBMi
    return 0;
#endif
    if (thread_cputime(-1, &tc) != 0) {
      UNREACHABLE();
    }
  }
#endif

Another question is why AIX doesn't use thread_cputime() internally to implement clock_gettime(CLOCK_THREAD_CPUTIME_ID) :-)
msg368947 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-15 14:23
time.thread_time() is documented as:
"Return the value (in fractional seconds) of the sum of the system and user CPU time of the current thread. It does not include time elapsed during sleep."
https://docs.python.org/dev/library/time.html#time.thread_time

So we must use utime+stime, just not stime (current PR 19381 implementation).

test_time.test_thread_time() validates that time.thread_time() doesn't include time spend during a sleep.
msg369027 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-16 09:39
New changeset 45410862321ae509e8753f239b0ea28fdcef5bad by Batuhan Taskaya in branch 'master':
bpo-40192: Use thread_cputime for time.thread_time to improve resolution (GH-19381)
https://github.com/python/cpython/commit/45410862321ae509e8753f239b0ea28fdcef5bad
msg369033 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python triager) Date: 2020-05-16 10:06
Should we mention about AIX support in availability section @vstinner? https://docs.python.org/3/library/time.html#time.thread_time
msg369035 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-16 10:12
I close the issue, since the commit was merged.

"Availability: Windows, Linux, Unix systems supporting CLOCK_THREAD_CPUTIME_ID." covers AIX. But you can add AIX if you consider that it's not explicit enough.
msg369036 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python triager) Date: 2020-05-16 10:14
> "Availability: Windows, Linux, Unix systems supporting CLOCK_THREAD_CPUTIME_ID." covers AIX.
That was my thought too, thanks again!
History
Date User Action Args
2020-05-16 10:14:23BTaskayasetmessages: + msg369036
2020-05-16 10:12:51vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg369035

stage: patch review -> resolved
2020-05-16 10:06:38BTaskayasetmessages: + msg369033
2020-05-16 09:39:12vstinnersetmessages: + msg369027
2020-05-15 14:23:24vstinnersetmessages: + msg368947
2020-05-15 14:20:56vstinnersetmessages: + msg368946
2020-05-15 14:02:26BTaskayasetmessages: + msg368945
2020-05-15 03:01:01vstinnersetnosy: + vstinner
messages: + msg368921
2020-04-05 01:23:19BTaskayasetnosy: + belopolsky, p-ganssle
2020-04-05 01:19:33BTaskayasetkeywords: + patch
stage: patch review
pull_requests: + pull_request18744
2020-04-05 01:15:32BTaskayacreate