Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop CALL_PROFILE special build? #72985

Closed
vstinner opened this issue Nov 25, 2016 · 5 comments
Closed

Drop CALL_PROFILE special build? #72985

vstinner opened this issue Nov 25, 2016 · 5 comments
Labels
3.7 (EOL) end of life

Comments

@vstinner
Copy link
Member

BPO 28799
Nosy @vstinner, @jeremyhylton
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • remove_call_profile.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2016-11-28.17:23:31.944>
    created_at = <Date 2016-11-25.09:02:53.660>
    labels = ['3.7']
    title = 'Drop CALL_PROFILE special build?'
    updated_at = <Date 2019-06-26.16:01:51.385>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2019-06-26.16:01:51.385>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2016-11-28.17:23:31.944>
    closer = 'vstinner'
    components = []
    creation = <Date 2016-11-25.09:02:53.660>
    creator = 'vstinner'
    dependencies = []
    files = ['45637']
    hgrepos = []
    issue_num = 28799
    keywords = ['patch']
    message_count = 5.0
    messages = ['281687', '281860', '281862', '281863', '346646']
    nosy_count = 3.0
    nosy_names = ['vstinner', 'Jeremy.Hylton', 'python-dev']
    pr_nums = ['552']
    priority = 'normal'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue28799'
    versions = ['Python 3.7']

    @vstinner
    Copy link
    Member Author

    Python/ceval.c contains conditional code to compute statistics on function calls when CALL_PROFILE is defined.

    Extract of Misc/SpecialBuilds.txt:

    CALL_PROFILE
    \------------
    
    Count the number of function calls executed.
    
    When this symbol is defined, the ceval mainloop and
    helper functions count the number of function calls
    made.  It keeps detailed statistics about what kind of
    object was called and whether the call hit any of the
    special fast paths in the code.
    

    Statistics can later be collected by sys.callstats().

    I'm unable to find any unit test on this feature. The feature was added in Python 2.3.1 by the changeset 16856c9514e0 in 2003:
    ---
    changeset: 27712:16856c9514e0
    branch: legacy-trunk
    user: Jeremy Hylton <jeremy@alum.mit.edu>
    date: Wed Feb 05 23:13:00 2003 +0000
    files: Include/ceval.h Include/compile.h Misc/SpecialBuilds.txt Python/ceval.c Python/compile.c Python/sysmodule.c
    description:
    Small function call optimization and special build option for call stats.

    -DCALL_PROFILE: Count the number of function calls executed.

    When this symbol is defined, the ceval mainloop and helper functions
    count the number of function calls made. It keeps detailed statistics
    about what kind of object was called and whether the call hit any of
    the special fast paths in the code.

    Optimization:

    When we take the fast_function() path, which seems to be taken for
    most function calls, and there is minimal frame setup to do, avoid
    call PyEval_EvalCodeEx(). The eval code ex function does a lot of
    work to handle keywords args and star args, free variables,
    generators, etc. The inlined version simply allocates the frame and
    copies the arguments values into the frame.

    The optimization gets a little help from compile.c which adds a
    CO_NOFREE flag to code objects that don't have free variables or cell
    variables. This change allows fast_function() to get into the fast
    path with fewer tests.

    I measure a couple of percent speedup in pystone with this change, but
    there's surely more that can be done.
    ---

    The changeset adds an optimization using CO_NOFREE and the CALL_PROFILE feature.

    My problem is that with my work on FASTCALL, it became harder to track where the functions are called in practice. It maybe out of the Python/ceval.c file. I'm not sure that statistics are still computed correctly after my FASTCALL changes, and I don't know how to check it.

    Python has already sys.setprofile(), cProfile and profile modules. There is also sys.settrace(). Do we still need CALL_PROFILE?

    Attached patch removes the feature:

    • Calling the the untested and undocumented sys.callstats() function now emits a DeprecationWarning warning
    • Remove the PyEval_GetCallStats() function and its documentation

    PyEval_GetCallStats() seems to be part of the stable API, but I don't expect that anyone uses it outside the CPython source code since it requires to rebuild CPython with a special build flag (-D CALL_PROFILE).

    @vstinner vstinner added the 3.7 (EOL) end of life label Nov 25, 2016
    @vstinner
    Copy link
    Member Author

    Jeremy Hylton, the author of the feature, approved the removal of CALL_PROFILE:
    https://mail.python.org/pipermail/python-dev/2016-November/146866.html

    Raymond Hettinger is also ok to remove it:
    "This seems reasonable to me. I've never used or needed this special build; StackOverflow has no mention of it; and a Google search comes up nearly empty. That said, it might be worthwhile to check with Jeremy to get his thoughts before removing his code."
    https://mail.python.org/pipermail/python-dev/2016-November/146864.html

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 28, 2016

    New changeset 5aa2171ee43f by Victor Stinner in branch 'default':
    Remove CALL_PROFILE special build
    https://hg.python.org/cpython/rev/5aa2171ee43f

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 28, 2016

    New changeset 07d8272d61e7 by Victor Stinner in branch 'default':
    Issue bpo-28799: Update Misc/SpecialBuilds.txt
    https://hg.python.org/cpython/rev/07d8272d61e7

    @vstinner
    Copy link
    Member Author

    I removed the function from Python 3.9 in bpo-37414.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant