New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bring test.support docs up to date #55224
Comments
The test.support docs are there to help CPython devs with writing good unit tests more easily. There are a few additions we've made in recent years that haven't made it into the .rst file, so it is easy to miss useful tools if you don't go looking through the module source code. There are some other helper modules (such as test.script_helper) that could also stand to be made easier to find. Fixing this is just a matter of doing a pass through test.support and the test directory looking for things that might be worth mentioning in the test package docs. |
I don't know if it matters much, but there's a slight mismatch in the description of test.support.verbose. The documentation says it's a boolean, while it's 0 or 1 in reality. Can it just be changed to True/False in the code of test.support and test.regrtest? It appears that all the other flags are True/False and there's no reason to keep this 0/1 (which is probably a relic from a distant past) |
Here's a patch fixing the 0/1 to True/False in a couple of places in test.support and test.regrtest I ran the test suite and it passes. A review is needed to commit. I'll keep working on the documentation itself in the meantime. |
Here's a patch to Doc/library/test.rst with additional several exported functions documented. These are the ones I found most important and clear. fcmp() is currently not documented (pending discussion in pydev). |
verbose isn't a boolean at all - it's an integer. You can supply it multiple times to bump the logging level up even higher than normal. ~/devel/py3k/Lib/test$ grep "verbose >" *.py (I didn't check explicitly, but I believe those verbose values are shorthand references to test.support.verbose) |
Nick, agreed regarding verbose. Somehow I didn't think it would be used in other modules like that, and only looked in regrtest where I didn't see anything meaningful about verbose not being binary. It's a good thing to document it, then :) |
Following the python-dev discussion, attaching a patch for removing fcmp and replacing its uses with assertAlmostEqual when needed. All tests pass and patchcheck is clean. Please review before I commit. |
The divmod() part of the patch is wrong: assertAlmostEqual does not support tuple arguments. The test succeeds because it first does an exact equality check, which apparently is true on your platform. But if it wasn't, you'd get TypeErrors. Also, fcmp() has a different definition of what "almost equal" means, but I assume this has been regarded in the discussion. |
Georg, Good catch on the tuples in assertAlmostEqual, thanks! I see three methods of resolution:
Any ideas? I can also commit (2) for the time being since it's the least dubious, and leave the resolution on 1 & 3 for later (and/or in separate issues). Regarding fcmp being not exactly assertAlmostEqual - it was discussed. |
(2) would be my choice. (1) *should* be true, but this is a change in the test semantics. (3) would be feature creep and I don't think it's a good idea. |
Attaching a revised patch with (2) implemented. |
Yes, that looks good now. |
Thanks. fcmp & FUZZ removal committed in revision 88558 |
Hi all, I just gave a look to the doc patch and it seems fine (it also applies without any warning on default). Eli, do you want to expand this patch further (and how :) or do you think it's still the version you want to commit? Can a core devel, then, give this patch a deeper look? |
This is an improvement that I think should be committed before 3.2.1. +.. function:: run_doctest(module, verbosity=None) + If *verbosity* is :const:`None`, :meth:`doctest` is run with verbosity set Should :meth:`doctest` be :func:`testmod` ? " If optional argument verbosity is not specified (or is None), pass The problem with the rewrite is that the keyword param of testmod is 'verbose', not 'verbosity'. 'Verbosity' is a dummy name used to either pass support.verbose to verbose, or not. So testmod is, in net effect, run with verbose=verbose or verbose=None. My attempt to explain a bad design (with probable markup errors): "If *verbosity* is :const:`None`, :func:`testmod` is run with verbose set to :data:`support.verbose`, which is set by :func:`regrtest`. Otherwise, it is run with verbose set to :const:`None` and subsequently replaced by :code:`'-v' in sys.argv`." +.. function:: temp_umask(umask) "sets the process umask to *umask*." ? +.. function:: find_unused_port(family=socket.AF_INET, socktype=socket.SOCK_STREAM) + Either this method or :func:`bind_port` should be used for any tests This is copied from the doc string but does really tell me which to use in which of the two situations. Other additions look OK to me. Some copied docstrings (or comments). Some are new. Support.py could also use a patch to add missing docstings (and turn a couple of comments into docstrings). |
I will review this again in a couple of days and will commit. |
I left a few comments on rietveld for the testdoc and remove_fcmp patches. |
Terry, I've incorporated your suggestions except the new formulation for verbosity to doctest.testmod, since it's not really clear which one is more accurate. I think it's intelligible as it is now (especially given that test.support is for use of Python contributors, and not random users). If you have strong preferences about it, feel free to commit another formulation. Ezio, I answered your comments in the code review tool. ---- I will commit a fixed patch to default. I see no reason to backport this since test.support is just a tool for core-devs and contributors and is mainly used for future developments. |
New changeset 2fd435ac3551 by Eli Bendersky in branch 'default': |
Can this issue be closed? |
There are still functions that are not documented, so I think this should stay open until we documented them (unless they shouldn't be documented -- in that case it can be closed). |
Ezio, you're right, there are still a lot of undocumented symbols (functions, classes, globals). I compiled a list of those I could not find in the .rst docs, excluding ones that are only used in regrtest.py itself: ------------------------------------------ get_attribute ------------------------------------------ |
Documenting script_helper.py would be good too. I seem to remember some discussion about merging it with support.py -- if that happens it can be documented there, otherwise a separate section/file should be created. |
I have created a pull request adding documentation for all of test.support, including script_helper.py. I hope this will be helpful. |
I merged Cheryl's PR to the dev branch, and triggered the backport to 3.7. If nobody beats me to it, I'll merge the latter tomorrow :) |
I have left comments to PR. |
Thanks Nick for merging. Should I open a new PR to address Serhiy's review or fix them on the original PR? |
Since Nick already merged and backported, at least a new PR is needed. Since this is an old issue with a long discussion and several inactives on the nosy list, I suggest a new issue starting from the merge result as a base: "More revisions to test.support docs". Make a list of Serhiy's questions and comments. TESTFN_NONASCII - How different from TESTFN_UNICODE? Add Nick, Serhiy, and me as nosy. We can discuss there how to handle the additional issues. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: