If the test suite is run with -3, many deprecation warnings are
reported. Quite a few are generated by code in the tests themselves,
but many are from constructs in the stdlib which are different or no
longer supported in 3.x.

Even aside from the fact that many of these warnings being caused by
code which I think many people would agree is not ideal, I think it is
important for these warnings to be fixed for Python 2.7. Since the 3.x
porting guidelines recommend that projects first run their test suite
with -3, these warnings are going to be generating extra noise for
developers to filter through when they try porting their code to 3.x.
They shouldn't need to wade through stdlib warnings to find warnings
about their own code.

Attached is a log of a complete test run with -3.

I don't think it's so important that tests not raises -3 warnings, but
that the stdlib doesn't.

I agree with Benjamin. We shouldn't release 2.7 w/ any of the standard
library itself generating a Py3kWarning, but that should not apply to the
test suite.

I have made this a release blocker until we can create separate issues for
the modules that are emitting the warning.

FWIW I tried to filter out the test-related warnings using the following
command line, but it didn't work (nothing was filtered out):

$ ./python -3 -W 'ignore::DeprecationWarning:test:0' -m test.regrtest

I've fixed a bunch of them: aifc (r75407), test_atexit (r75408), bsddb
(r75409), test_calendar (r75410), StringIO (r75411), socket (r75412),
sndhdr (r75413), test_memoryio (r75415), test_profilehooks (r75417),
test_random (r75419), httplib (r75420), uuid (r75421).

Here is a record of stderr after a new regrtest run.

(interestingly, one of the culprits for py3k warnings is lib2to3)

Making this a deferred blocker so as not to clutter 2.6.4 release.

Many fixes of modules and packages outside Lib/test

Dev notes:

• for bsddb and dbhash the warning message should include "module" to
  be ignored by "test_support.import_module"
• patch for mailbox is copied from Lib/cgi.py
• other fixes are trivial

Patch for the compiler package and astgen.py tool.

Basically:
def __init__(self, (left, right), lineno=None):

==>

  def __init__(self, leftright, lineno=None):

Patch for pprint: when keys are not the same type, "-3" emits warnings.

Partial backport from Py3k.

In issue7092_lib_many_fixes.diff: why did you change some of the warning messages?

In issue7092_pprint.diff: I'm not sure these changes are right, because they could modify existing behaviour. Raymond, what do you think?

Updated "lib_many_fixes" patch:

• added filterwarning for pprint
• added fixes for Lib/sunau.py and Lib/unittest/case.py

Ready for review and merge:

• issue7092_lib_many_fixes_v2.diff
• issue7092_compiler.diff

One thing: "lambda k: k in selfdata" can be replaced with
"selfdata.__contains__". It may have better performance characteristics
too.

I learned something.

Looks good, thank you.

Both patches have been committed in r77310.

Still some "-3" warnings to silence:

./python -3m "collections" >/dev/null
./python -3c "import Cookie"
./python -3c "import idlelib.rpc"
./python -3c "import logging.handlers"
./python -3c "import multiprocessing"
./python -3c "import shelve"
./python -3c "import trace"

All these modules yield:
"DeprecationWarning: the cPickle module has been removed in Python 3.0"

There's also "bsddb" but it is deprecated.

All these modules yield:
"DeprecationWarning: the cPickle module has been removed in Python 3.0"

I think this warning is both annoying (cPickle is a legitimate module to use in 2.x, since pickle is much slower) and useless (2to3 should be able to do the module rename -- Benjamin, does it?). Therefore I suggest to remove this warning.

I think this warning is both annoying (cPickle is a legitimate module to use in 2.x, since pickle is much slower) and useless (2to3 should be able to do the module rename -- Benjamin, does it?). Therefore I suggest to remove this warning.

You may be right, but keep in mind that cPickle and pickle are only *mostly* compatible with each other. There are uses of one which will not automatically work if you switch to the other.

+1 to remove this boring cPickle message.

cStringIO do not print such messages.

Antoine> (2to3 should be able to do the module rename -- Benjamin, does it?).
2to3 will rename cPickle to pickle.

The Big Patch™.

It removes 99% of noisy deprecation messages.

test_exceptions and test_pep352 fixed in r77913.

test_unittest fixed in r77911. See also bpo-7837.

In issue7092_test_support:

1. silence_py3k_warning should have a docstring that explain what the function does and what filters should be, possibly including an example;
2. I'd use "if not filters:" instead of "if filters == ():";
3. are there cases where the third arg of filterwarnings is not DeprecationWarning? If not, it's probably better to turn "filters" in a list of regex instead than a sequence of tuples like (regex, DeprecationWarning);
4. maybe silence_py3k_warning could yield an instance of WarningsRecorder, in case someone wants to access the silenced warnings.

Docstring added.

Notes:

• there's 2 categories of -3 warnings: SyntaxWarning and DeprecationWarning.
• AFAICT, the WarningsRecorder needs "catch_warnings(record=True)".
  And in such case, *all* warnings are silenced.

Applied the syntax fixes in r77942.

The enhancement proposed on bpo-7849 should be considered before reviewing the last 2 patches.

If bpo-7849 is accepted, these patches will need update.

Silence the last warnings, using check_py3k_warnings.

But we could provide a smaller patch, if bpo-7832 is fixed before.

Checking the pybsddb 4.8.4devel with current trunk I still see this:

"""
Berkeley DB 4.8.26: (December 18, 2009)
bsddb.db.version(): (4, 8, 26)
bsddb.db.__version__: 4.8.4devel1
bsddb.db.cvsid: $Id: _bsddb.c,v 6408a812f035 2010/03/15 19:15:06 jcea $
py module: /home/pybsddb/build/lib.solaris-2.10-i86pc-2.7/bsddb3/init.pyc
extension module: /home/pybsddb/build/lib.solaris-2.10-i86pc-2.7/bsddb3/init.pyc
python version: 2.7a4+ (trunk, Mar 16 2010, 00:50:35)
[GCC 4.4.3]
My pid: 5068
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
............................................................................................................................................................................................................................................................................................................................................................................................................................................./usr/local/lib/python2.7/threading.py:575: DeprecationWarning: sys.exc_clear() not supported in 3.x; use except clauses
self.__exc_clear()
.........................................
----------------------------------------------------------------------
Ran 470 tests in 7.605s

OK

"""

Python 2.6.4 has the same issue too.

Checking the pybsddb 4.8.4devel with current trunk I still see this:

It is because the "test_early_close" module calls resetwarnings().
It defeats the filter in the "threading" module.

You may change the test case with a context manager:

  with warnings.catch_warnings():
      warnings.resetwarnings()
      # test the warning

I took part of Florent latest patch, changed a few things and committed it in r79023.

Florent you are right, good catch. I can not use context managers because "with" is invalid syntax in python 2.3 and 2.4, that I must support for a while.

I have implemented a context manager manually, following the description in PEP-343. This change is available in pybsddb 4.8.3+. I plan to integrate pybsddb 4.8.3+, see bpo-8156.

More warnings silenced with r79165.

other py3k warnings silenced with r79187 r79188 r79189

Now the "-Wd" and "-3" flags are active on some buildbots.
And the first failures ...

Need to backport fixes from py3k branch for:

• cgitb "SyntaxWarning: tuple parameter unpacking has been removed in 3.x"
• idlelib "reduce() not supported in 3.x; use functools.reduce()"

And test_multiprocessing is failing on: "sys.exc_clear() not supported in 3.x; use except clauses"

Other py3k warnings:

lib\distutils\util.py:209: DeprecationWarning: apply() not supported in 3.x; use func(*args, **kwargs)
return apply(os.path.join, paths)

lib\distutils\command\build_ext.py:679: DeprecationWarning: apply() not supported in 3.x; use func(*args, **kwargs)
return apply(os.path.join, ext_path) + '_d' + so_ext

lib\platform.py:1035: DeprecationWarning: dict.has_key() not supported in 3.x; use the in operator
if _default_architecture.has_key(sys.platform):

Fixed with r79558, except test_multiprocessing.

Additional fixes: r79576, r79577, r79578 (idlelib package and some test modules)

This issue is over, except for bpo-7723 which is tracked separately.

Barry, is it OK if I backport this to 2.6 before the rc1?

Hi Ezio, what's the status on this issue for 2.6.6rc1?

I am removing this as a release blocker given all of Ezio's great work to get the test suite clean with -3. I will leave it up to him to actually close the issue once the work is complete. It no longer needs to block 2.6.6.

I fixed the deprecation warnings on 2.6 in r83519, r83576-r83579, r83620-r83622, r83627-r83632, r83635, r83637-83642, r83652-83656, r83658, r83687.
Now the test suite should pass with and without -3 -Wd with no errors and no warnings (except an import warning on _ctypes and/or _multiprocessing).
This can be closed.