Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhanced hotshot profiler with high-resolution timer #46471

Closed
MrJean1 mannequin opened this issue Mar 2, 2008 · 10 comments
Closed

Enhanced hotshot profiler with high-resolution timer #46471

MrJean1 mannequin opened this issue Mar 2, 2008 · 10 comments
Labels
extension-modules C modules in the Modules dir type-feature A feature request or enhancement

Comments

@MrJean1
Copy link
Mannequin

MrJean1 mannequin commented Mar 2, 2008

BPO 2218
Nosy @freddrake, @birkenfeld, @gpshead
Files
  • hires_hotshot.tgz
  • hires_hotshot2.tgz
  • hires_hotshot3.tgz
  • hires_hotshot4.tgz
  • hires_hotshot5.tgz
  • hires_hotshot_2.6.diffs.tgz
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2010-08-21.23:57:05.325>
created_at = <Date 2008-03-02.23:45:10.674>
labels = ['extension-modules', 'type-feature']
title = 'Enhanced hotshot profiler with high-resolution timer'
updated_at = <Date 2010-08-21.23:57:05.323>
user = 'https://bugs.python.org/MrJean1'

    bugs.python.org fields:

    activity = <Date 2010-08-21.23:57:05.323>
actor = 'georg.brandl'
assignee = 'none'
closed = True
closed_date = <Date 2010-08-21.23:57:05.325>
closer = 'georg.brandl'
components = ['Extension Modules']
creation = <Date 2008-03-02.23:45:10.674>
creator = 'MrJean1'
dependencies = []
files = ['9588', '9606', '9635', '9644', '9664', '9815']
hgrepos = []
issue_num = 2218
keywords = ['patch']
message_count = 10.0
messages = ['63194', '63224', '63268', '63373', '63407', '63429', '63485', '64335', '74466', '114614']
nosy_count = 4.0
nosy_names = ['fdrake', 'georg.brandl', 'gregory.p.smith', 'MrJean1']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue2218'
versions = ['Python 2.7']
    @MrJean1
    Copy link
    Mannequin Author

    MrJean1 mannequin commented Mar 2, 2008

    The _hotshot module uses the gettimeofday function to profile the run
    time.

    I enhanced the hotshot module in Python 2.5.2 to use a high resolution
    timer where available (RDTSC on x86/_64, MFTB/U on PowerPC and gethrtime
    on Solaris).

    The improved hotshot module has been tested on 32-bit MacOS X 10.4.11
    Tiger (Intel) and 10.3.9 Panther (PPC), on 64-bit RHEL 3u7 (Opteron) and
    on 64-bit Solaris 10.

    The 3 modified files are Modules/_hotshot.c, Lib/hotshot/log.py and
    Lib/hotshot/stats.py are attached.

    @MrJean1 MrJean1 mannequin added extension-modules C modules in the Modules dir type-feature A feature request or enhancement labels Mar 2, 2008
    @MrJean1
    Copy link
    Mannequin Author

    MrJean1 mannequin commented Mar 3, 2008

    Do *not* use the attached files. A fix is forthcoming. My apologies.

    @MrJean1
    Copy link
    Mannequin Author

    MrJean1 mannequin commented Mar 4, 2008

    Attached are the improved hotspot files, rev 2. The changes are rather
    drastic, in particular since hires time delta's may exceed 32-bit int.

    These 3 files have been tested with Python 2.5.2 only and on 32-bit
    Linux and MacOS X and on 64-bit Linux and Solaris 10. They do pass the
    hotspot and other tests.

    However, no other platforms or compilers have been tested. Except for
    64-bit PPC, the code should fall back to the previous behavior using
    gettimeofday.

    @MrJean1
    Copy link
    Mannequin Author

    MrJean1 mannequin commented Mar 8, 2008

    Here is another version of the profiler files with 7 minor changes in the
    _hotshot.c file. Use this hires_version3 set, if any.

    @MrJean1
    Copy link
    Mannequin Author

    MrJean1 mannequin commented Mar 8, 2008

    There are 7 other, potentially serious issues in the original _hotshot.c
    file. All those are being fixed for the next version of the enhanced
    hotshot files.

    The 7 issues are:

    1. functions flush_data and do_stop may create an infinite recursion on
      line 567.

    2. The pack_string and pack_add_info functions do not check for string
      lengths exceeding BUFFERSIZE and may corrupt memory in that case.

    3. In line 1182, linetimings should be frametimngs:
      {"frametimings", T_LONG, offsetof(ProfilerObject, linetimings),
      READONLY},

    4. Coverage members frametimings, linetimings and lineevents are set on
      line 1562 ff. after different values may have been written into the log
      file header by function write_header on lines 1440 ff.

    5. The coverage__doc__ on line 1546 is incomplete.

    6. At several places, errors are detected but not reported and not sets.

    7. An int value is passed where a long is specified or expected. T_LONG
      should be T_INT on lines 1182-1184 and there are several calls to PyInt_FromLong with an int argument on line 487 ff. and other places.

    @MrJean1
    Copy link
    Mannequin Author

    MrJean1 mannequin commented Mar 9, 2008

    Attached is the latest, hopefully final version of the enhanced and
    improved hotshot profiler files. In addition to fixes for the 7 issues
    mentioned previously, type Py_ssize_t is used instead of size_t. Use
    this verion 4 and discard all previous ones.

    Please review this code in file _hotshot.c and let me know of any rule
    or guideline violations. I am more than happy to correct any mistakes.

    @MrJean1
    Copy link
    Mannequin Author

    MrJean1 mannequin commented Mar 12, 2008

    Attached is yet another, final update of the enhancements to the hotshot
    profiler. It includes modifications of all 3 files, Modules/_hotshot.c,
    Lib/hotshot/log.py and Lib/hotshot/stats.py.

    @MrJean1
    Copy link
    Mannequin Author

    MrJean1 mannequin commented Mar 22, 2008

    Here are the forward diffs of the 3 files against the version in the
    trunk.

    @freddrake
    Copy link
    Member

    May I ask what the reasoning is for further developing hotshot as part
    of the core? My understanding, based on discussions on python-dev, is
    that hotshot is being deprecated in favor of cProfile.

    If hotshot still provides functionality that's interesting (and it may),
    perhaps it makes sense to maintain and improve it in a separate
    distribution.

    @birkenfeld
    Copy link
    Member

    hotshot won't get any more updates.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    extension-modules C modules in the Modules dir type-feature A feature request or enhancement
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @birkenfeld @freddrake