New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unittest runner needs an option to call gc.collect() after each test #62108
Comments
It would be nice if there was a command-line option to tell the test runner to call gc.collect() after each test. See https://groups.google.com/d/msg/python-tulip/tstjvOQowIU/IRihc8NCHZUJ -- Glyph points out that always doing this is problematic because it slows down tests too much; OTOH it's a useful option to have in case you're tracking down something that happens (or doesn't happen) when an object is collected (e.g. in __del__). |
See also bpo-17534. |
IMO this is a good reason to implement your specific tearDown method (or call addCleanup if you prefer), possibly in a shared base class. I don't think this is a good candidate for a command-line option, it's too specialized and it's also not something which should be enabled blindly. In other words, each test case should know whether it needs a collection or not. |
On Sun, May 5, 2013 at 4:29 AM, Antoine Pitrou <report@bugs.python.org> wrote:
This is not for tests that know or expect they need a call to It's rare that extra collections cause tests to fail (and if it does |
I'm afraid I'm inclined to agree with Antoine. This seems a relatively specialised problem and I'd expect to see it solved in the test system rather than in unittest itself. One solution, and something I'd like to see, is a way for test systems to extend the unittest command line handling so that they can add extra options. At the moment it's very painful to do. |
TBH I would *love* for there to be a standardized way to extend the command line options! If it existed I would help myself. As it is, that's way to complicated. (Then again, most customizations to the default test runner are too complicated IMO. :-) |
The default test runner is quite horrible. |
+1. I found myself wanting to add a "-F" option à la regrtest, and I |
Is this enhancement still open? I've run into this problem previously, and would be more than happy to implement this feature. |
Does the cpython test runner have this? Might be nice there. Not sure if On Monday, April 27, 2015, Ned Deily <report@bugs.python.org> wrote:
|
I still think that it's something that people can trivially implement and that has no place in the standard unittest package. |
It's trivial to add to a single test or even a single TestCase subclass, yes. The use case is more that you have a ton of tests and you suspect there's a problem due to GC. Being able to call gc.collect() after each test through the flip of a flag would be useful. But I agree it's iffy whether this belongs in the unittest package; we could add it to the CPython run_tests.py script, or you could request this as a feature from py.test. If someone wants to add this to run_tests.py I think this issue would be the place to review the patch. |
See also bpo-23839 where proposed not only call gc.collect(), but clear all caches in test.regrtest. |
I'm going to disagree with michael and antoine here. The *internals* should be clean and pluggable for sure, but this is actually a pretty common thing to try, so there's no reason to force it to only be done by external plugins. Right now the way to plug this in has been complicated by the addition of module / class suites, which already perform extra work around individual tests, but in a non-introspectable / extensible fashion. So you could add this as a hook to the loader (decorate each test with some new thing) and a CLI option to use that hook for a gc collect call. Alternatively, we could face down the class/module stuff and rearrange it to be extensible (e.g. something along the lines of testresources internals - generic groups pre-post stuff) or via some interaction with TestResult.... but I really dislike using TestResult to control the run - I have a better layout mostly sketched in mind but haven't had time to formalise it. So I recommend the TestLoader hook point - its well within the current responsibilities for it to do this, and I don't see any long term maintenance issues. |
Le 28/04/2015 21:00, Robert Collins a écrit :
Can I take this as meaning you're not opposed to adding other options to |
"say, something to run a test until failure, or to I think a few complimentary things. unittest extensability currently requires a new CLI entry point for each thing. I'd like to fix that. The actual plumbing is fairly extensible, though its not always obvious. I'd like to fix that too - without any global state getting involved. Of the things you mention, running a given command line until failure and checking for reference leaks are both straight forward, very common requests (as is the gc check) and I'd like to see those implemented as extensions shipped in the stdlib. Running in parallel becomes important when one is doing slow (e.g. functional) tests with unittest, and I think thats important to support. It is however much harder to do well: some of the current idioms that have snuck in (like the handling of stdout/stderr capturing without the buffer flag) are not well matched to the needs of reporting on concurrent tests to users. I'm not in any way opposed to a good implementation, but it would need to be good I think - there's not much point having a poor implementation, given the rich set of parallel test runners that are out there that already build on the unittest core (green, nose, testrepository just for starters). The only unique audience for stdlib test facilities is the stdlib itself and I think a better way to solve that is to enable the use of alternative runners for our own tests (moving them to be a little cleaner should enable that) - and then the point where it matters is really 'when buildbots would be enough faster to make a difference'. |
Any comments about this proposed patch? |
While I like the idea of adding an option to force a garbage collection after running each time, I don't like how unittest is growing :-( It looks more and more like regrtest and I hate regrtest (I reworked its code recently to create a less ugly "libregrtes"). Once Antoine Pitrou proposed to add "plugins" to unittest. I agree that we need a gate between unittest and all other test frameworks: nose, py.test, testtools/testrepository/testr, etc. Seriously, don't you think that something is wrong with such API? def __init__(self, module='__main__', defaultTest=None, argv=None,
testRunner=None, testLoader=loader.defaultTestLoader,
exit=True, verbosity=1, failfast=None, catchbreak=None,
buffer=None, warnings=None, *, tb_locals=False,
gc_enabled=False): |
Well, maybe we can add yet another parameter to all unittest classes |
This issue is no newcomer friendly, I remove the "easy" keyword. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: