Title: New command line option supported by unittest.main() for running initialization code before tests
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.4, Python 3.5
Status: open Resolution:
Dependencies: Superseder:
Assigned To: michael.foord Nosy List: chris.jerdonek, ezio.melotti, michael.foord, piotr.dobrogost, vinay.sajip, Никита Конин
Priority: normal Keywords:

Created on 2012-10-24 19:58 by piotr.dobrogost, last changed 2017-06-20 09:03 by Никита Конин.

Messages (12)
msg173699 - (view) Author: Piotr Dobrogost (piotr.dobrogost) Date: 2012-10-24 19:58
When running tests bundled with a library it would be useful to be able to run some initialization code before tests are being run so that one can configure options pertaining to logging for example.

For this reason it would be useful to have a new command line option handled by unittest.main() for running custom Python module before running tests.

msg173862 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-10-26 17:20
Can you give an example of what you would like the API to look like (i.e. what you would like to be able to type from the command-line if, say, configuring logging as in the SO post)?

I'm not sure I understand why a wrapper script isn't a good enough solution, as described in the accepted answer on SO.
msg173863 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2012-10-26 17:22
I would do this with a module level (global) flag as to whether the initial set up has been done and a BaseTestCase class that checks this flag and calls a set up function if it hasn't been done yet (and then sets the flag).
msg173917 - (view) Author: Piotr Dobrogost (piotr.dobrogost) Date: 2012-10-27 10:50

The example given by Holger Krekel ( showing how it can be done with pytest is exactly the thing I had in mind. It would be good to have this feature incorporated into unittest.


I guess checking this flag must be protected by a lock in case test cases are run in parallel. If so wouldn't this be kind of a bottleneck? Is this flag necessary? Can't we just assume that initialization code had already run when any testcase runs?

There's interesting question how would initialization code be run in case of auto discovery. In the simplest form it would be run just once before all modules with tests are run. But it would be interesting to allow running initialization per test module. To support this unittest should somehow pass name of test module to be executed to initialization code. Alternatively there could be some special named init files which would be run before running test modules.
msg173925 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2012-10-27 12:27
Yes, protecting the set up with a lock would be easy, although unittest doesn't directly support running tests in parallel anyway.
msg174071 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-10-28 18:50
Is there any common use case other than logging? For example, logging could be covered by an additional command-line argument for unittest.main(), e.g.

--logconfig /path/to/logging-config.json

which would cause unittest.main() to load the JSON in the specified file (expected to be a suitable dict) and pass it to logging.config.dictConfig().

That would handle a perhaps not uncommon use case without the user having to write any code, but rather just provide a suitable logging configuration.
msg174079 - (view) Author: Piotr Dobrogost (piotr.dobrogost) Date: 2012-10-28 21:07
I don't see other use cases but maybe guys behind py.test, where there is such a feature could come up with other use cases.
The solution with passing config dict in a new command line parameter seems to be enough as long as logging is concerned. However it's not as versatile as custom initialization code of course.
msg174082 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2012-10-28 22:20
I always solve the logging problem with test-development-production configuratins and ensuring that running tests imports the logging setup. I've never wanted / needed to configure logging from the command line.
msg174091 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-10-29 00:38
@Michael: When unit testing one's own projects, I agree with you. But I think Piotr's use case is when you download some new library from PyPI or clone it from GitHub/BitBucket/etc., and you want to run tests on that code without changing any of it, as you're just exploring at this point. Now ideally you'd just be able to do e.g.

python test --logconfig filename.json

or, if it isn't set up to do that, at least look for a test folder with scripts that have 'if __name__ == "__main__": unittest.main()', which I can then run. In either of these cases, even if the library I'm testing uses logging, I can't readily make use of it to look at DEBUG messages, say, unless I write some code (not much code, I grant you).

Of course, once you get to the point where you've decided to use that library in your project, you would integrate its logging setup with the logging setup of your own code, in your tests.
msg174107 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2012-10-29 11:03
Is this logging specific issue an actual problem or a theoretical problem? It's not a problem I've ever seen (myself), nor the one the issue was opened for.
msg174110 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-10-29 12:12
Not an actual problem (for me), and although the issue was opened for more general functionality, logging is the only specific use case that was actually cited.
msg175590 - (view) Author: Piotr Dobrogost (piotr.dobrogost) Date: 2012-11-14 20:48
> But I think Piotr's use case is when you download some new library (...)

Yes, indeed.
Date User Action Args
2017-06-20 09:03:43Никита Конинsetnosy: + Никита Конин
2012-11-14 20:48:07piotr.dobrogostsetmessages: + msg175590
2012-10-29 12:12:13vinay.sajipsetmessages: + msg174110
2012-10-29 11:03:26michael.foordsetmessages: + msg174107
2012-10-29 00:38:25vinay.sajipsetmessages: + msg174091
2012-10-28 22:20:10michael.foordsetmessages: + msg174082
2012-10-28 21:07:41piotr.dobrogostsetmessages: + msg174079
2012-10-28 18:50:50vinay.sajipsetnosy: + vinay.sajip
messages: + msg174071
2012-10-27 12:27:52michael.foordsetmessages: + msg173925
2012-10-27 10:50:08piotr.dobrogostsetmessages: + msg173917
2012-10-26 17:22:44michael.foordsetassignee: michael.foord
2012-10-26 17:22:31michael.foordsetmessages: + msg173863
2012-10-26 17:20:47chris.jerdoneksetnosy: + chris.jerdonek
messages: + msg173862
2012-10-26 16:30:34ezio.melottisetnosy: + ezio.melotti, michael.foord
2012-10-24 19:58:35piotr.dobrogostcreate