classification
Title: regrtest.py -R not working
Type: behavior Stage: patch review
Components: Tests Versions: Python 3.0, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, christian.heimes, michael.foord, ocean-city, rhettinger (5)
Priority: high Keywords patch

Created on 2008-03-03 16:27 by ocean-city, last changed 2009-04-28 00:14 by ajaksu2.

Files
File name Uploaded Description Edit Remove
regrtest.patch belopolsky, 2008-03-03 18:55 diff against revision 61203
doctest_twice.zip ocean-city, 2008-03-04 09:03
fix_test_collections.patch ocean-city, 2008-03-04 10:39
Messages (14)
msg63215 - (view) Author: Hirokazu Yamamoto (ocean-city) Date: 2008-03-03 16:27
Sorry, I don't have patch, but regrtest.py -R not working now.

E:\python-dev\trunk\Lib\test>py25 regrtest.py -R ::
test_grammar
test_grammar skipped -- No module named _abcoll

E:\python-dev\trunk\Lib\test>py regrtest.py -R ::
test_grammar
test_grammar skipped -- cannot import name _Abstract

E:\python-dev\trunk\Lib\test>py3k regrtest.py -R ::
  File "regrtest.py", line 174
    print __doc__
                ^
SyntaxError: invalid syntax
msg63220 - (view) Author: Hirokazu Yamamoto (ocean-city) Date: 2008-03-03 17:50
Sorry, I was stupid. I ran test in wrong directory.
release25-maint runs fine, and other two has same
error.

test_grammar
test_grammar skipped -- cannot import name _Abstract
msg63221 - (view) Author: Christian Heimes (christian.heimes) Date: 2008-03-03 17:53
I'm going to work on the issue later
msg63222 - (view) Author: Alexander Belopolsky (belopolsky) Date: 2008-03-03 18:03
It looks like you are running regrtest from the trunk with the py3k
interpretor.

Works for me.
msg63223 - (view) Author: Hirokazu Yamamoto (ocean-city) Date: 2008-03-03 18:17
Yes, I did mistake first, but py3k fails even in correct directory.

E:\python-dev\py3k\Lib\test>py3k --version
Python 3.0a3+

E:\python-dev\py3k\Lib\test>py3k regrtest.py -R :: test_os.py
test_os
test_os skipped -- cannot import name _Abstract
1 test skipped:
    test_os
1 skip unexpected on win32:
    test_os
[43267 refs]

abc._Abstract seems to be gone at r61098, but regrtest.py still
references to it.
msg63225 - (view) Author: Alexander Belopolsky (belopolsky) Date: 2008-03-03 18:55
attached simple patch seems to fix the problem but should be reviewed by
Christian.
msg63227 - (view) Author: Christian Heimes (christian.heimes) Date: 2008-03-03 19:23
It seems r61204 has fixed the bug. Can you test it on your machine
please? My old laptop is too slow.
msg63228 - (view) Author: Alexander Belopolsky (belopolsky) Date: 2008-03-03 19:56
> Can you test it on your machine please?

Running on a 4-core Opteron (2.6GHz).

Should complete in an hour or so ...
msg63229 - (view) Author: Alexander Belopolsky (belopolsky) Date: 2008-03-03 20:02
test_collections fails (with -R :: only).  Does not look like something
related to the recent changes, rather test is not happy about being
repeated.

$ ./python Lib/test/regrtest.py -R ::  test_collections 
test_collections
beginning 9 repetitions
123456789
test test_collections failed -- Traceback (most recent call last):
  File "Lib/doctest.py", line 2131, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for collections.namedtuple
  File "Lib/collections.py", line 13, in namedtuple

..
msg63231 - (view) Author: Alexander Belopolsky (belopolsky) Date: 2008-03-03 20:59
Here are the results from regrtest.py -R ::

..
301 tests OK.
7 tests failed:
    test_collections test_cprofile test_frozen test_inspect
    test_logging test_pkg test_profile
..
$ cat reflog.txt
test_cmd_line leaked [-23, 0, 0, 23] references, sum=0
test_compiler leaked [0, 0, 18, 0] references, sum=18
test_deque leaked [100, 100, 100, 100] references, sum=400
test_ftplib leaked [0, 172, -6, -166] references, sum=0
test_heapq leaked [108, 130, 121, 114] references, sum=473
test_itertools leaked [7380, 7380, 7380, 7380] references, sum=29520
test_list leaked [50, 50, 50, 50] references, sum=200
test_set leaked [680, 680, 680, 680] references, sum=2720
test_smtplib leaked [0, 0, -86, 86] references, sum=0
test_threading leaked [0, 86, -86, 86] references, sum=86
test_urllib2_localnet leaked [3, 3, 3, 3] references, sum=12
test_userlist leaked [50, 50, 50, 50] references, sum=200

Do buildbots run -R regressions?
msg63244 - (view) Author: Hirokazu Yamamoto (ocean-city) Date: 2008-03-04 04:27
I can see test_collections.py failure even on r61203
(just before Christian's commit) - r61098 (removed abc._Abstract)

E:\python-dev\trunk\Lib\test>py regrtest.py -R :: test_collections.py
test_collections
beginning 9 repetitions
123456789
test test_collections failed -- Traceback (most recent call last):
  File "e:\python-dev\trunk\lib\doctest.py", line 2131, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for collections.namedtuple
  File "e:\python-dev\trunk\lib\collections.py", line 13, in namedtuple

----------------------------------------------------------------------
File "e:\python-dev\trunk\lib\collections.py", line 16, in
collections.namedtupl
e
Failed example:
    Point = namedtuple('Point', 'x y')
Exception raised:
    Traceback (most recent call last):
      File "e:\python-dev\trunk\lib\doctest.py", line 1231, in __run
        compileflags, 1) in test.globs
      File "<doctest collections.namedtuple[0]>", line 1, in <module>
        Point = namedtuple('Point', 'x y')
    NameError: name 'namedtuple' is not defined
(snip)

I'll run py regrtest.py -R :: on r61203 - r61098 and compare its
result to Alexander's result, but my machine is damn slow, probably
it'll take long time.
msg63248 - (view) Author: Hirokazu Yamamoto (ocean-city) Date: 2008-03-04 09:03
>I'll run py regrtest.py -R :: on r61203 - r61098 and compare its
>result to Alexander's result, but my machine is damn slow, probably
>it'll take long time.

Sorry, I abandoned this at test_compiler. Please forget this sentense...

Anyway, Christian's fix greatly works for me. Thanks!

>E:\python-dev\trunk\Lib\test>py regrtest.py -R :: test_collections.py

For this error, I could write a minimam code to reproduce it.
Please expand the attached zip file into lib/test and run a.py.
You'll see similar error.
msg63250 - (view) Author: Hirokazu Yamamoto (ocean-city) Date: 2008-03-04 10:39
I wrote simple patch to workaround this problem.
(avoid to reuse DocTestSuite)

# To my eyes, doctest.DocTestSuite(module=collections)
# and test_support.run_doctest(collections, verbose)
# are doing same test???
msg63275 - (view) Author: Hirokazu Yamamoto (ocean-city) Date: 2008-03-05 06:15
I did more investigation.
Failure on second DocTestSuite run happens because
lib/doctest.py (2107) test.globs.clear()
is executed after first test runs.

I don't know if this is bug or not. Document in
doctest.py says test.globs will be untouched on
failure but become empty on success, but this
behavior prevents reusage of DocTestSuite.
History
Date User Action Args
2009-04-28 00:14:12ajaksu2setnosy: + michael.foord

stage: patch review
2008-12-05 09:59:01georg.brandllinkissue2447 superseder
2008-03-17 21:34:40rhettingersetassignee: rhettinger ->
2008-03-05 06:15:27ocean-citysetmessages: + msg63275
2008-03-04 16:27:20rhettingersetassignee: rhettinger
nosy: + rhettinger
2008-03-04 10:39:30ocean-citysetfiles: + fix_test_collections.patch
messages: + msg63250
2008-03-04 09:03:05ocean-citysetfiles: + doctest_twice.zip
messages: + msg63248
2008-03-04 04:27:16ocean-citysetmessages: + msg63244
2008-03-03 20:59:45belopolskysetmessages: + msg63231
2008-03-03 20:02:10belopolskysetmessages: + msg63229
2008-03-03 19:56:40belopolskysetmessages: + msg63228
2008-03-03 19:23:31christian.heimessetmessages: + msg63227
2008-03-03 18:55:03belopolskysetfiles: + regrtest.patch
keywords: + patch
messages: + msg63225
2008-03-03 18:17:20ocean-citysetmessages: + msg63223
2008-03-03 18:03:01belopolskysetnosy: + belopolsky
messages: + msg63222
2008-03-03 17:53:33christian.heimessetpriority: high
nosy: + christian.heimes
messages: + msg63221
2008-03-03 17:50:02ocean-citysetmessages: + msg63220
versions: - Python 2.5
2008-03-03 16:27:08ocean-citycreate