This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients Mark.Shannon, hroncok, petr.viktorin, rhettinger, vstinner
Date 2021-05-10.15:43:58
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1620661438.18.0.44030531836.issue43760@roundup.psfhosted.org>
In-reply-to
Content
I understand that some projects manually call the profile and/or trace functions, and temporarily set use_tracing 0 while calling these functions.

Some projects restore use_tracing to the correct value (compute the efficient value), some projects simply set use_tracing to 1.

I see 3 use cases:

* disable tracing temporarily (set use_tracing to 0)
* reenable tracing (compute use_tracing to the correct value)
* check if tracing is used (get use_tracing)

We can add 3 functions:

* PyThreadState_DisableTracing()
* PyThreadState_EnableTracing()
* PyThreadState_GetTracing()

PyThreadState_EnableTracing(tstate) would do something like:

    tstate->cframe->use_tracing = (tstate->c_tracefunc || tstate->c_profilefunc);

If we added these functions, I can then add an implementation for Python 3.9 and older to my https://github.com/pythoncapi/pythoncapi_compat project for backward compatibility.

The problem is that some projects also increase temporarily ts->tracing. Since I would like to make PyThreadState opaque, I would prefer to hide this access behind a function call as well. Maybe we need an API to call profile and/or trace functions?

--

According to the bugzilla compiler errors:

> greenlet: https://bugzilla.redhat.com/show_bug.cgi?id=1957784

It has already been fixed:

* https://github.com/python-greenlet/greenlet/commit/6c5f0963eb00eeb1cfb337c6edbd3efaf7d8eacc
* https://github.com/python-greenlet/greenlet/commit/352b974447bb489cb2778e07c3832d0cc60e0e4a

It uses:

* "tstate->use_tracing = 0;"
* "tstate->use_tracing = (tstate->tracing <= 0 && (...)"

> dipy: https://bugzilla.redhat.com/show_bug.cgi?id=1958203

It uses:

* "tstate->use_tracing = 0;"
* "tstate->use_tracing = 1;"
* "tstate->use_tracing = (tstate->c_profilefunc || (...)"
* "return tstate->use_tracing && retval;"
* "if (tstate->use_tracing) {"

> yappi: https://bugzilla.redhat.com/show_bug.cgi?id=1958896

* "ts->use_tracing = 1;"
* "ts->use_tracing = 0;"

> smartcols: https://bugzilla.redhat.com/show_bug.cgi?id=1958938

It uses "tstate->use_tracing = 0;".

> scikit-learn: https://bugzilla.redhat.com/show_bug.cgi?id=1958976

It uses:

* "tstate->use_tracing = 0;"
* "tstate->use_tracing = 1;"

> The usage comes from https://github.com/cython/cython/blob/master/Cython/Utility/Profile.c

Simplified code:

--------------
static int __Pyx_TraceSetupAndCall(...)
{
    ...
    tstate->tracing++;
    tstate->use_tracing = 0;

    if (tstate->c_tracefunc)
        retval = tstate->c_tracefunc(tstate->c_traceobj, *frame, PyTrace_CALL, NULL) == 0;
    if (retval && tstate->c_profilefunc)
        retval = tstate->c_profilefunc(tstate->c_profileobj, *frame, PyTrace_CALL, NULL) == 0;

    tstate->use_tracing = (tstate->c_profilefunc ||
                           (CYTHON_TRACE && tstate->c_tracefunc));
    tstate->tracing--;
    ...
}

  int __Pyx_use_tracing = 0;

  #define __Pyx_TraceCall(funcname, srcfile, firstlineno, nogil, goto_error)             \
  if (nogil) {                                                                           \
      if (CYTHON_TRACE_NOGIL) {                                                          \
          PyThreadState *tstate;                                                         \
          PyGILState_STATE state = PyGILState_Ensure();                                  \
          tstate = __Pyx_PyThreadState_Current;                                          \
          if (unlikely(tstate->use_tracing) && !tstate->tracing &&                       \
                  (tstate->c_profilefunc || (CYTHON_TRACE && tstate->c_tracefunc))) {    \
              __Pyx_use_tracing = __Pyx_TraceSetupAndCall(&$frame_code_cname, &$frame_cname, tstate, funcname, srcfile, firstlineno);  \
          }                                                                              \
          PyGILState_Release(state);                                                     \
          if (unlikely(__Pyx_use_tracing < 0)) goto_error;                               \
      }                                                                                  \
  } else {                                                                               \
      PyThreadState* tstate = PyThreadState_GET();                                       \
      if (unlikely(tstate->use_tracing) && !tstate->tracing &&                           \
              (tstate->c_profilefunc || (CYTHON_TRACE && tstate->c_tracefunc))) {        \
          __Pyx_use_tracing = __Pyx_TraceSetupAndCall(&$frame_code_cname, &$frame_cname, tstate, funcname, srcfile, firstlineno);  \
          if (unlikely(__Pyx_use_tracing < 0)) goto_error;                               \
      }                                                                                  \
  }
--------------
History
Date User Action Args
2021-05-10 15:43:58vstinnersetrecipients: + vstinner, rhettinger, petr.viktorin, Mark.Shannon, hroncok
2021-05-10 15:43:58vstinnersetmessageid: <1620661438.18.0.44030531836.issue43760@roundup.psfhosted.org>
2021-05-10 15:43:58vstinnerlinkissue43760 messages
2021-05-10 15:43:58vstinnercreate