This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Should PyObject_Call() call the profiler on C functions, use C_TRACE() macro?
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: jdemeyer, methane, ppperry, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017-02-08 15:39 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
profiler_c_code.py vstinner, 2017-02-09 11:05
Messages (7)
msg287343 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-08 15:39
call_function() and do_call_core() functions of Python/ceval.c use C_TRACE() macro to call the profiler, but PyObject_Call() and similar functions like _PyObject_FastCallKeywords() don't.

Is it a bug or a deliberate choice for performance?

Since PyObject_Call() and similar functions have now fast paths, I don't think that it's still needed to have these fast paths in ceval.c too. But I kept fast paths in ceval.c because of C_TRACE().
msg287402 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-09 11:05
Output of attached profiler_c_code.py example:
---
haypo@selma$ ./python profiler_c_code.py 
(<frame object at 0xf9d788>, 'call', None)
(<frame object at 0xf9d788>, 'return', 'h')
(<frame object at 0xf9d788>, 'call', None)
(<frame object at 0xf9d788>, 'return', 'e')
(<frame object at 0xf9d788>, 'call', None)
(<frame object at 0xf9d788>, 'return', 'l')
(<frame object at 0xf9d788>, 'call', None)
(<frame object at 0xf9d788>, 'return', 'l')
(<frame object at 0xf9d788>, 'call', None)
(<frame object at 0xf9d788>, 'return', 'o')
(<frame object at 0xf9d498>, 'c_call', <built-in function setprofile>)
---

Except of sys.setprofile(), the profiler doesn't see any C calls! list() and map() are simply "hidden".

So tools like coverage miss a lot of C call? Again, is it a compromise between performance and correctness, or just a regular bug?
msg322810 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2018-07-31 19:26
I always assumed that enabling profiling only from the Python bytecode interpreter was a deliberate choice.
msg322833 - (view) Author: (ppperry) Date: 2018-08-01 02:19
issue30990 is related
msg322838 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-08-01 04:17
FYI, _lsprof uses PyObject_Call()
https://github.com/python/cpython/blob/ea68d83933e6de6cabfb115ec1b8888301947369/Modules/_lsprof.c#L120-L123

If PyObject_Call() trigger profiler, lsprof should avoid recursion.
msg322925 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2018-08-02 07:52
I would prefer to wait with this issue until PEP 580 has been decided. If it's accepted, it will change function calling code a lot. As I wrote in PEP 580, I was planning to have a PEP on profiling anyway.
msg402392 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-21 22:20
This issue has no activity for 3 years and was created 4 years ago. I close it.
History
Date User Action Args
2022-04-11 14:58:43adminsetgithub: 73688
2021-09-21 22:20:13vstinnersetstatus: open -> closed
resolution: out of date
messages: + msg402392

stage: resolved
2018-08-02 07:52:08jdemeyersetmessages: + msg322925
2018-08-01 04:17:06methanesetmessages: + msg322838
2018-08-01 02:19:56ppperrysetnosy: + ppperry
messages: + msg322833
2018-07-31 19:26:45jdemeyersetnosy: + jdemeyer
messages: + msg322810
2017-02-09 11:05:29vstinnersetfiles: + profiler_c_code.py

messages: + msg287402
2017-02-08 15:39:38vstinnercreate