This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: [subinterpreters] Improve test coverage for subinterpreters
Type: Stage: test needed
Components: Subinterpreters, Tests Versions: Python 3.6, Python 3.5
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eric.snow, grahamd, ncoghlan, petr.viktorin, skrah
Priority: normal Keywords:

Created on 2015-07-03 00:30 by eric.snow, last changed 2022-04-11 14:58 by admin.

Messages (14)
msg246107 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-07-03 00:30
We do very little testing of subinterpreters in CPython.  About all I'm aware of is in test_tracemalloc.  I'll be working on improving test coverage as a precursor to fixing some existing bugs.
msg246149 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-07-03 08:10
There are some basic tests in test_capi as well:

msg246207 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-07-03 19:17
FTR, subinterpreters are already accessible during testing with _testcapi.run_in_subinterp().


That function is used here:

* Lib/test/
* Lib/test/ (run_in_subinterp()) is used here:

* Lib/test/
* Lib/test/
* Lib/test/
msg246208 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-07-03 19:21
Also, I was mistaken about test_tracemalloc.
msg246237 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-07-04 01:22
Now I'm wondering what further test coverage we really need...
msg246238 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-07-04 04:10
Sorry about the misleading reference to tracemalloc in my email - it was actually test_atexit I was debugging in the PEP 432 branch. tracemalloc only came up in that context because automatically skips subinterpreter tests when tracemalloc is running. From the function comments:

# Issue #10915, #15751: PyGILState_*() functions don't work with
# sub-interpreters, the tracemalloc module uses these functions internally

As far as test coverage goes, what I would actually like to do is to run regrtest itself in a subinterpreter, and blacklist tests until it passes. This would double the length of a test run, so hiding it behind a resource option would probably be a good idea.
msg246253 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-07-04 12:33
> Now I'm wondering what further test coverage we really need...

Ideally we'd test every C module with the tests executing "in
parallel" (sort of) in multiple interpreters.

I have done so for _decimal, which is mostly okay due to the
thread-local contexts. However, there are a couple of minor

The reason I haven't pursued this is that *if* you run _decimal
in multiple interpreters, you get nasty sounding (but harmless)
warnings from libmpdec, which complains about being initialized
multiple times.

So from the absence of bug reports I concluded that no one is
using the sub-interpreter feature with _decimal.
msg246255 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-07-04 13:20
Adding Petr Viktorin to the nosy list as well, as the kind of issues Stefan mentions there are the kinds of things that the PEP 489 extension module import issues *didn't* address yet, but we'd like to address in the 3.6 iteration of the multi-phase initialisation support for extension modules.

Petr, for context, this issue is about improving our test coverage for subinterpreter support - see the earlier messages for more details.
msg246256 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-07-04 13:30
I think for fast access we need a hybrid solution that allows
static types (heap types slowed down _decimal) *and* cache the
thread local values (like it's currently done for the thread-local
context in _decimal).

Caching the context brought an enormous speedup (about 20%).

I'm not sure if all that can be done in a general API, but it's
clearly the fastest way.
msg246257 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-07-04 13:51
That's jumping ahead a little - this issue is only about running the regression test suite from a subinterpreter, and establishing what *already* works properly. Dealing with the fallout of any quirks and outright failures we discover that way will be a different issue :)
msg246298 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-07-05 09:53
Nick, this may be a misunderstanding, but you mentioned issues
that PEP 489 did not address yet.  The only issue left for _decimal
is the speed issue.  Sub-interpreter tests run fine even with the
existing module state API, *if* one is willing to take a speed hit.

[I get that this is outside the scope of this issue, just for
msg246677 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-07-13 01:52
As a possible starting point for this, I'll point to

Now, I'd expected running that in a process that was *already* running regrtest to fail miserably (it would stomp all over itself).

But what we could potentially do is launch a *clean* subprocess, where the only thing it did was:

    from _testcapi import run_in_subinterp

    regrtest_in_subinterpreter = (
    from test import regrtest

I'd currently expect that to fail as well, but I think the failures might be enlightening :)

I'm also not sure we need to integrate this directly into the main regrtest test runner - it could just be a separate submodule invoked like "python -m test.subinterpretertest".

If we later decided to integrate it, then it could go behind a "-usubinterpreter" resource that invoked "test.subinterpretertest" in a subprocess.
msg246695 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-07-13 17:12
+1 for python -m test.subinterpretertest. Based on my experiments,
most tests will either fail or leak memory at the very least.
msg319559 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2018-06-14 22:24
Note that bpo-32604 is strongly related.
Date User Action Args
2022-04-11 14:58:18adminsetgithub: 68741
2020-05-15 00:41:59vstinnersetcomponents: + Subinterpreters
title: improve test coverage for subinterpreters -> [subinterpreters] Improve test coverage for subinterpreters
2018-06-14 22:24:48eric.snowsetmessages: + msg319559
2015-07-13 17:12:41skrahsetmessages: + msg246695
2015-07-13 01:52:52ncoghlansetmessages: + msg246677
2015-07-05 09:53:01skrahsetmessages: + msg246298
2015-07-04 13:51:12ncoghlansetmessages: + msg246257
2015-07-04 13:30:48skrahsetmessages: + msg246256
2015-07-04 13:20:03ncoghlansetnosy: + petr.viktorin
messages: + msg246255
2015-07-04 12:33:05skrahsetnosy: + skrah
messages: + msg246253
2015-07-04 04:10:16ncoghlansetstatus: pending -> open
nosy: + grahamd
messages: + msg246238

2015-07-04 01:22:27eric.snowsetstatus: open -> pending

messages: + msg246237
2015-07-03 19:21:27eric.snowsetmessages: + msg246208
2015-07-03 19:17:20eric.snowsetmessages: + msg246207
2015-07-03 08:10:52ncoghlansetmessages: + msg246149
2015-07-03 00:30:03eric.snowcreate