Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

time.thread_time isn't outputting in nanoseconds in AIX #84373

Closed
isidentical opened this issue Apr 5, 2020 · 9 comments
Closed

time.thread_time isn't outputting in nanoseconds in AIX #84373

isidentical opened this issue Apr 5, 2020 · 9 comments
Labels
3.9 only security fixes stdlib Python modules in the Lib dir

Comments

@isidentical
Copy link
Sponsor Member

BPO 40192
Nosy @abalkin, @pitrou, @vstinner, @pganssle, @isidentical
PRs
  • bpo-40192: Use thread_cputime for time.thread_time to improve resolution #19381
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-05-16.10:12:51.343>
    created_at = <Date 2020-04-05.01:15:32.204>
    labels = ['library', '3.9']
    title = "time.thread_time isn't outputting in nanoseconds in AIX"
    updated_at = <Date 2020-05-16.10:14:23.164>
    user = 'https://github.com/isidentical'

    bugs.python.org fields:

    activity = <Date 2020-05-16.10:14:23.164>
    actor = 'BTaskaya'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-05-16.10:12:51.343>
    closer = 'vstinner'
    components = ['Library (Lib)']
    creation = <Date 2020-04-05.01:15:32.204>
    creator = 'BTaskaya'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 40192
    keywords = ['patch']
    message_count = 9.0
    messages = ['365807', '368921', '368945', '368946', '368947', '369027', '369033', '369035', '369036']
    nosy_count = 5.0
    nosy_names = ['belopolsky', 'pitrou', 'vstinner', 'p-ganssle', 'BTaskaya']
    pr_nums = ['19381']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue40192'
    versions = ['Python 3.9']

    @isidentical
    Copy link
    Sponsor Member Author

    The resolution for thread_time is really low on AIX, but fortunately there is a way to get thread time in nanoseconds with thread_cputime.

    -bash-4.4$ ./python
    Python 3.9.0a5+ (heads/master:909f4a3, Apr  4 2020, 20:15:24) [C] on aix
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import time
    >>> time.thread_time()
    0.02

    @isidentical isidentical added 3.9 only security fixes stdlib Python modules in the Lib dir labels Apr 5, 2020
    @vstinner
    Copy link
    Member

    Which implementation is currently used on AIX? What's the output of the following command?

    $ ./python
    Python 3.9.0a6+ (heads/master:4a12d12186, May 15 2020, 04:55:17) 
    >>> import time; time.get_clock_info('thread_time')
    namespace(adjustable=False, implementation='clock_gettime(CLOCK_THREAD_CPUTIME_ID)', monotonic=True, resolution=1e-09)

    @isidentical
    Copy link
    Sponsor Member Author

    current:
    >>> import time
    >>> import time
    >>> time.get_clock_info('thread_time')
    namespace(adjustable=False, implementation='clock_gettime(CLOCK_THREAD_CPUTIME_ID)', monotonic=True, resolution=0.01)
    >>> time.thread_time()
    0.07
    
    PR 19381:
    >>> import time
    >>> time.get_clock_info('thread_time')
    namespace(adjustable=False, implementation='thread_cputime()', monotonic=True, resolution=1e-09)
    >>> time.thread_time()
    0.002379953

    @vstinner
    Copy link
    Member

    I found this documentation on AIX thread_cputime():
    https://www.ibm.com/support/knowledgecenter/ssw_aix_71/t_bostechref/thread_cputime.html

    """
    Syntax

    #include <sys/thread.h>
    int thread_cputime (tid, ctime)
    tid_t tid;
    thread_cputime_t * ctime ;
    typedef struct {
        uint64_t utime; /* User time in nanosenconds */
        uint64_t stime; /* System time in nanoseconds */
    } thread_cputime_t;

    Description

    The thread_cputime subroutine allows a thread to query the CPU usage of the specified thread (tid) in the same process or in another process. If a value of -1 is passed in the tid parameter field, then the CPU usage of the calling thread is retrieved.

    CPU usage is not the same as the total life of the thread in real time, rather it is the actual amount of CPU time consumed by the thread since it was created. The CPU usage retrieved by this subroutine contains the CPU time consumed by the requested thread tid in user space (utime) and system space (stime).

    The thread to be queried is identified using the kernel thread ID which has global scope. This can be obtained by the application using the thread_self system call. Only 1:1 thread mode is supported. The result for M:N thread mode is undefined.

    The CPU usage of a thread that is not the calling thread will be current as of the last time the thread was dispatched. This value will be off by a small amount if the target thread is currently running.
    """

    Ok good, it returns the user time *and* the system time, and it's the thread CPU time.

    So it sounds reasonable to use it to implement time.thread_time().

    By the way, the v8 project calls thread_cputime() on AIX when clock_gettime(CLOCK_THREAD_CPUTIME_ID) is requested, also to get better resolution:
    https://github.com/v8/v8/blob/a5038c42283a09f65c44229907123e15a779feb7/src/base/platform/time.cc#L68

    // On AIX clock_gettime for CLOCK_THREAD_CPUTIME_ID outputs time with
    // resolution of 10ms. thread_cputime API provides the time in ns
    #if defined(V8_OS_AIX)
      thread_cputime_t tc;
      if (clk_id == CLOCK_THREAD_CPUTIME_ID) {
    #if defined(__PASE__)  // CLOCK_THREAD_CPUTIME_ID clock not supported on IBMi
        return 0;
    #endif
        if (thread_cputime(-1, &tc) != 0) {
          UNREACHABLE();
        }
      }
    #endif

    Another question is why AIX doesn't use thread_cputime() internally to implement clock_gettime(CLOCK_THREAD_CPUTIME_ID) :-)

    @vstinner
    Copy link
    Member

    time.thread_time() is documented as:
    "Return the value (in fractional seconds) of the sum of the system and user CPU time of the current thread. It does not include time elapsed during sleep."
    https://docs.python.org/dev/library/time.html#time.thread_time

    So we must use utime+stime, just not stime (current PR 19381 implementation).

    test_time.test_thread_time() validates that time.thread_time() doesn't include time spend during a sleep.

    @vstinner
    Copy link
    Member

    New changeset 4541086 by Batuhan Taskaya in branch 'master':
    bpo-40192: Use thread_cputime for time.thread_time to improve resolution (GH-19381)
    4541086

    @isidentical
    Copy link
    Sponsor Member Author

    Should we mention about AIX support in availability section @vstinner? https://docs.python.org/3/library/time.html#time.thread_time

    @vstinner
    Copy link
    Member

    I close the issue, since the commit was merged.

    "Availability: Windows, Linux, Unix systems supporting CLOCK_THREAD_CPUTIME_ID." covers AIX. But you can add AIX if you consider that it's not explicit enough.

    @isidentical
    Copy link
    Sponsor Member Author

    "Availability: Windows, Linux, Unix systems supporting CLOCK_THREAD_CPUTIME_ID." covers AIX.
    That was my thought too, thanks again!

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants