New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make test.script_helper more comprehensive, and use it in the test suite #53726
Comments
test.script_helper has a couple of dedicated functions to launch a Python interpreter instance in a subprocess. Unfortunately, it is little used and most test modules use their own ad hoc calls to subprocess instead. Remedying the situation would require:
This was suggested by Nick in bpo-477863. |
(The email daemon was not in a happy place, so posting directly) On Thu, Aug 5, 2010 at 8:50 AM, Antoine Pitrou <report@bugs.python.org> wrote:
script_helper just factored out the old test_cmd_line approach which |
One other feature for the new-and-improved helpers: add a flag to allow "-E" to be omitted (as per the comment in test_cmd_line) |
I still think this is a good idea, I'm just not actively working on it. It might make a good project for someone wanting to get to know the process of working on CPython without having to deal with anything that is particularly tricky to understand. |
That sounds exactly like me :) I can have a look at this ticket. |
I just tried using script_helper in a new test, so I have a couple of comments. I don't see stdout and stderr being conflated, it looks to me like they are returned separately, at least by the assert methods. The assert methods return results, which is unlike other assert methods. This is very useful, even essential, and I wouldn't want to give it up. That conflicts with the current unittest conventions, though. It would be a big help if 'err' were returned with the refcount line removed if it is there, which would make tests using the methods return the same 'err' regardless of whether they are run under a debug build or not. I think the names of the two assert functions should follow the current unit test conventions (assertPythonRunOK and asssertPythonRunNotOK, perhaps?) |
That's because I wrote the assert methods since this issue was opened :)
Indeed.
Well, they are functions, not methods, so I don't think they have to |
OK, fine on the convention, but I'd still like a more memorable name for assert_python_failure. I've been working on this issue off and on today, and I've had to look up that name at least four times. I can remember assert_python_ok, but I can't remember whether its inverse is assert_python_fails, assert_python_bad, or what. For some reason I haven't guessed 'failure' even once so far :) (I know it's not assert_python_not_ok because I remember it isn't parallel...) |
Here is a patch that causes _assert_python to remove the refcount lines from stderr. |
Hmm. Having posted that it occurs to me that it could be useful to have the _remove_refcount function in test.support as remove_refcount instead. |
There's already strip_python_stderr() :) |
Oh, good, I'll use that then. I could have sworn I looked for that functionality a couple weeks ago and couldn't find it. |
This is already used in Lib/test/script_helper.py:42, and a way to call _assert_python() without using -E has also been introduced0. The next step would be to check if these functions could be used elsewhere in the other tests. Also note that these functions are not currently documented anywhere. |
@rodrigue did you ever make any progress with this? |
@mark, I had a look at the time I commented on the ticket but didn't get a chance to progress much in the end. |
Script helpers has been "made more discoverable" by moving it into the test.support namespace, but it has not been documented in the test.support docs. I can no longer remember if this is intentional or not. Regardless, this issue is still valid insofar as there are probably a number of places in the test suite where script_helpers can be used that they aren't being used. We probably don't want a multi-module changeset, though, so this could become a meta issue for new issues for converting particular test files to use script_helpers. |
Hello @r.david.murray!
Does this mean you'd like to leave this issue open until all of the test files are updated in separate issues? Or is it that you'd like separate changesets for each module, but all associated with this issue? Thanks! |
It could be done either way, but I suspect that indovidual issues for test module changes, entered as dependencies on this issue, is probably the more effective in this case. |
Okay thanks @r.david.murray, I'll take a look at doing this. I'll open an issue when I figure out which module to start on. |
Apparently I was imaginig that script_helpers had been moved into test.support. test.support was changed into a module in prep for doing so, but as far as I can see it was never moved. So that does remain to be done as well, and should be done first. |
Okee dokee, I'll tackle that first! |
I have a patch for moving script_helper as R. David suggested. Here it is. |
bobcatfish: Sorry, I didn't refresh and see your comment before submitting my patch. It only moves script_helpers, but doesn't address the original OP. Hopefully I made your life easier, not harder. |
re: test.test_tools.py - should this also move into test.support ? |
test_tools is a test suite that tests the tools in the scripts directory, it is not tools for testing. So no, it doesn't belong in test.support. |
Yes, I also agree that this looks better. But as this was first humble patch submission to CPython, I followed David Murry's instructions during my review - which was to move the entire module. I would be opposed to adding the methods directly into __init__.py, I would recommend still moving script_helper into test.support , and adding a list of symbols from script_helper.py into __init__ eg: test/support/init.py: test/some_test.py: I am currently in the PyCon CPython sprint room if you are also here. |
Uploading new patch that includes the creation of Lib/test/support/script_helper.py |
Sounds like a great plan to me @flipmcf! |
I actually originally wanted to do just that (move the script helpers routine into the old support.py) but I was in the minority. test.support was turned into a package exactly so that script_helpers could become a sub-namespace. As I remember, it was Nick Coghlan who championed the sub-namespace position :) |
namespaces are a honkin' great idea I don't have enough information to say that 'script_helper' is the right name for the namespace, tho. |
@r.david.murray and @flipmcf: I've audited all test modules using sys.executable for a starting point, and these tests look like they could benefit from helper methods to invoke python (probably the ones in script_helper, but we may want to create others if the use cases don’t quite fit):
@r.david.murray I'm going to start with test_unicodedata.py for a nice easy starting point so I'll create a separate issue for it. @flipmcf should I start using your patch set as-is or is there more to come? |
@Christie - Nope. This patch can stand on it's own, simply moving the package to where it belongs. In fact, I'd recommend applying the patch first. |
So @flipmcf we're sticking with the test.support.script_helper namespace? |
Alternatively, you can import public functions from support/script_helper.py in support/init.py. Then you can do from test.support import assert_python_ok |
Reviewing iss9517_move_script_helpers_py.patch:
|
@r.david.murray I've created bpo-23981, I don't think I have permissions to add it as a dependency to this issue. |
Hey @berker.peksag, @r.david.murray! Here's another change set where script_helpers.py is moved with an hg mv (still looks in the patch like the file was deleted and re-added, not sure if that's expected). I've ALSO removed some unused imports from script_helpers.py - temp_dir was being imported from script_helpers in a few places tho, so I had to fix that. If you guys would rather not include that change here, please let me know! |
Thanks for the patch, Christie. Review comments: http://bugs.python.org/review/9517/
I don't know if it's important. I'm using a very old version of Mercurial (2.0.2 :)), so it can be related to different Mercurial versions. Attaching a patch for reference. |
You should create the patch with "hg diff --git" to preserve moving. And then apply it with "hg import", not the patch command. Unfortunately Rietveld don't like patches in git format. |
Hey @berker.peksag, I've added a new patch in response to your review!
Ick, I'm guessing it's okay to just use plain old "hg diff" then, if Rietveld doesn't like the patches in git format. |
The committer should be careful, and manually make "hg mv" and apply the patch with the patch command, not "hg import". We shouldn't lost the history. |
@serhiy.storchaka - just double checking, do you guys need me to make any more changes to the patch? And is there any more review needed, or is it possible for this to be merged? Thanks very much! |
New changeset f65174aef9ea by Berker Peksag in branch 'default': |
Thanks very much @berker.peksag! |
Hey @berker.peksag, @r.david.murray, @serhiy.storchaka, If you get a chance I've got some changes up for review at: Thanks! |
Just noting that bpo-18576 has a draft patch for test.support.script_helper documentation (the "move to the support module" part of the current patch there is obsolete) |
Sure, that would be great. |
Created bpo-24279 for updating test_base64. |
Created bpo-24398 for updating test_capi. |
Berker committed the original patch to move the helper module, so adjusting the stage back to reflect the ongoing review on related issues. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: