Not sure how feasible this is, but when running tests under Windows it's very nice to run with --nowindows on, else you get barraged with a ton of debugging warnings. The problem is that if you use -j the use of --nowindows gets turned off. It would be nice if using --nowindows worked with -j to speed up test execution on Windows while also being less noisy.

I'd be happy to always disable assert dialogs/output in regrtest, and then use "-n" to enable stderr output. That way people simply testing their own debug builds don't think it's failed, and the buildbots keep all the output they currently have.

Steve's idea SGTM.

This issue has been fixed by the following change:

changeset:   98427:b48ae54ed5be
user:        Victor Stinner <victor.stinner@gmail.com>
date:        Wed Sep 30 01:39:28 2015 +0200
files:       Lib/test/libregrtest/cmdline.py Lib/test/libregrtest/main.py Lib/te
description:
Issue #25220, libregrtest: Call setup_python(ns) in the slaves

Slaves (child processes running tests for regrtest -jN) now inherit
--memlimit/-M, --threshold/-t and --nowindows/-n options.

* -M, -t and -n are now supported with -jN
* Factorize code to run tests.
* run_test_in_subprocess() now pass the whole "ns" namespace to the child
  process.

Steve wrote:
"I'd be happy to always disable assert dialogs/output in regrtest, and then use "-n" to enable stderr output. That way people simply testing their own debug builds don't think it's failed, and the buildbots keep all the output they currently have."

I would prefer to have a new option for that, to keep backward compatibility on regrtest parameters.

We may also always hide popups and deprecated --nowindows, but that's differrent.

What do you think?

So basically what I said (-n becomes redundant) and a new option for logging asserts to stderr? I don't really mind, but it seems like more of a compatibility break.

Steve Dower added the comment:
> So basically what I said (-n becomes redundant)

ok

> and a new option for logging asserts to stderr?

Yes. Modify PCbuild/rt.bat and/or Tools/buildbot/test.bat to enable
the option on Windows buildbots.

> Yes. Modify PCbuild/rt.bat and/or Tools/buildbot/test.bat to enable
> the option on Windows buildbots.

Wait, I'm not sure that I understood. You want to hide warnings by
default, and add an options to show them? If it's the case, no,
PCbuild/rt.bat and Tools/buildbot/test.bat must not be modified.
Sorry.

For popups: would it be useful for a specific use case to get them?
(Should we add an option to *enable* popups, since they will be
disabled by default.)

I was only going to disable them by default for regrtest. The only way users will see them is when using the debug binaries or in their own extensions built in debug mode. We just don't want them scaring users who have just built from source and don't know about -n when they run the tests.

Just to add my 0.02USD:

I think my preferred course of action would be to enable '-n' by default and deprecate/ignore the option.  For the assert output on stderr, I think it would be fine to hide it by default, but show it if verbose > i for some value of i (0 or 1, probably).  They really don't add much except spam in the buildbot logs.

Also, for the record, changes to how the buildbots are called isn't very hard; let me know if any are needed.  I'd like to eventually get rid of Tools/buildbot and just let the buildbots call the same things the same way users do.

Between Victor's fix and #23919, is there anything left to do for this?

We are good, I close the issue.