classification
Title: Test failures with COUNT_ALLOCS
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: 23034 Superseder:
Assigned To: serhiy.storchaka Nosy List: bkabrda, python-dev, rkuska, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2013-11-08 12:30 by bkabrda, last changed 2017-10-17 09:25 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
00141-fix-tests_with_COUNT_ALLOCS.patch bkabrda, 2013-11-08 12:30 review
00141-fix-tests_with_COUNT_ALLOCS-v2.patch bkabrda, 2014-01-10 10:25 review
00141-fix-tests_with_COUNT_ALLOCS-v3.patch bkabrda, 2014-12-11 11:22 review
00141-fix-tests_with_COUNT_ALLOCS-v4.patch serhiy.storchaka, 2014-12-11 20:55 review
00141-fix-tests_with_COUNT_ALLOCS-v5.patch serhiy.storchaka, 2014-12-13 17:47 review
Pull Requests
URL Status Linked Edit
PR 3910 closed vstinner, 2017-10-06 19:58
PR 3927 merged vstinner, 2017-10-09 09:21
Messages (19)
msg202414 - (view) Author: Bohuslav "Slavek" Kabrda (bkabrda) * Date: 2013-11-08 12:30
When Python is compiled with COUNT_ALLOCS, some tests in test_gc and test_module fail. I'm attaching the patch that skips 3 of them and modifies assertions in one of them, so that the tests pass.

I'm however still unsure about one of the skipped tests, since I'm unsure whether I totally understand what's wrong there - test_gc_ordinary_module_at_shutdown.

My guess is that due to COUNT_ALLOCS causing immortal types, the "final_a" and "final_b" types don't get destroyed on line [1] as they do in builds without COUNT_ALLOCS. AFAICS they are only "un-immortalized" on this line and destroyed during the following loop [2]. The problem here is that the order of destroyed modules is not deterministic, so sometimes the builtins module gets destroyed before the "final_X" and there is no "print" function, which makes the __del__ functions from "final_X" fail. IMO the best thing to do is to just skip this test with COUNT_ALLOCS. But I may be wrong, I don't have a great insight into Python's GC and module unloading.


[1] http://hg.python.org/cpython/annotate/0f48843652b1/Python/import.c#l383
[2] http://hg.python.org/cpython/annotate/0f48843652b1/Python/import.c#l394
msg202415 - (view) Author: Bohuslav "Slavek" Kabrda (bkabrda) * Date: 2013-11-08 12:30
And the patch...
msg207844 - (view) Author: Bohuslav "Slavek" Kabrda (bkabrda) * Date: 2014-01-10 10:25
Since 3.4.0.b2, this also causes failures in another tests: test_io, test_logging, test_threading, test_warnings. There are various cases testing that some types get collected when the interpreter shuts down.

I'm attaching a new patch that covers all of these.
msg207845 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-01-10 10:28
What is COUNT_ALLOCS?

Python 3.4 now has sys.getallocatedblocks() and a new tracemalloc module which are compiled by default.
msg207848 - (view) Author: Bohuslav "Slavek" Kabrda (bkabrda) * Date: 2014-01-10 10:33
As noted in Misc/SpecialBuilds:

COUNT_ALLOCS
------------

Each type object grows three new members:

    /* Number of times an object of this type was allocated. */
    int tp_allocs;

    /* Number of times an object of this type was deallocated. */
    int tp_frees;

    /* Highwater mark:  the maximum value of tp_allocs - tp_frees so
     * far; or, IOW, the largest number of objects of this type alive at
     * the same time.
     */
    int tp_maxalloc;

...
We use this for Fedora's python debug build to get some interesting debugging info. (If you try to "grep -r" through Python 3.4 source code, you'll find quite a few ifdefs with COUNT_ALLOCS.)
msg232450 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-10 22:40
Why do you check hasattr(sys, 'getrefcount') in test_io.py, but hasattr(sys, 'getcounts') in all other tests?
msg232469 - (view) Author: Bohuslav "Slavek" Kabrda (bkabrda) * Date: 2014-12-11 11:22
Good catch, using getrefcount was a mistake. I'm attaching a new version which always checks for getcounts (and also applies on 3.4.2).
msg232492 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-11 20:55
LGTM (only one nitpick -- there are trailing spaces in test_gc).

But there are other tests which are failed with COUNT_ALLOCS. Here is a patch with additional fixes for these tests in test_gc and test_warnings.
msg232615 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-13 17:47
Thenks Antoine for great idea proposed in comments on Rietveld. Following patch introduces strip_python_stdout() which strips COUNT_ALLOCS debug output from stdout (unfortunately this operation is not always unambiguous) and call it in assert_python_ok() and assert_python_failure(). This automatically fixes a large number of tests. Also fixed a number of other tests failing with COUNT_ALLOCS. Virtually all tests are now fixed except test_doctest and test_distutils.

A large part of the patch will be applied only to 3.4 and 2.7. With resolved issue23034 the patch for 3.5 will be much simpler, strip_python_stdout() and @requires_clean_stdout will gone away.
msg255697 - (view) Author: Robert Kuska (rkuska) * Date: 2015-12-02 10:03
With Python-3.5 and COUNT_ALLOCS enabled following new tests fail also:


FAIL: test_is_finalizing (test.test_sys.SysModuleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.5.0/Lib/test/test_sys.py", line 767, in test_is_finalizing
    self.assertEqual(stdout.rstrip(), b'True')
AssertionError: b'' != b'True'
---------------------------------

======================================================================
FAIL: test_print_traceback_at_exit (test.test_traceback.SyntaxTracebackCases)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.5.0/Lib/test/test_traceback.py", line 210, in test_print_traceback_at_exit
    self.assertEqual(stderr.splitlines(), expected)
AssertionError: Lists differ: [] != [b'Traceback (most recent call last):', b'[75 chars]ero']
Second list contains 3 additional elements.
First extra element 0:
b'Traceback (most recent call last):'
- []
+ [b'Traceback (most recent call last):',
+  b'  File "<string>", line 8, in __init__',
+  b'ZeroDivisionError: division by zero']
msg255698 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-12-02 10:28
COUNT_ALLOCS was added 22 years ago. I guess that the usecase is to track memory leaks, right?

branch:      legacy-trunk
user:        Sjoerd Mullender <sjoerd@acm.org>
date:        Mon Oct 11 12:54:31 1993 +0000
files:       Include/object.h Modules/arraymodule.c Modules/config.c.in Modules/imageop.c Modules/imgfile.c Objects/floatobject.c Objects/intobject.c Objects/listobj
description:
* Extended X interface: pixmap objects, colormap objects visual objects,
  image objects, and lots of new methods.
* Added counting of allocations and deallocations of builtin types if
  COUNT_ALLOCS is defined.  Had to move calls to NEWREF down in some
  files.
* Bug fix in sorting lists.
msg255700 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-12-02 10:33
00141-fix-tests_with_COUNT_ALLOCS-v5.patch: please don't do that! It makes tests much more verbose for a compilation option which is hidden and probably almost never used in the wild. The option has no configuration option for example.

*If* you really want to keep the feature, I would prefer to make it more visible (add a configuration option) and disable the output at exit by default. It's better to add a new "-X showcountallocs" as it was done with "-X showrefcount". Before, we had to fix a lot of unit tests (as 00141-fix-tests_with_COUNT_ALLOCS-v5.patch) to strip the "[xxx refs]" from stderr, it was very annoying.

"Python 3.4 now has sys.getallocatedblocks() and a new tracemalloc module which are compiled by default."

IMHO these two debug features superseded COUNT_ALLOCS. Please try to convince me of the use case of this very old debug feature.
msg255701 - (view) Author: Robert Kuska (rkuska) * Date: 2015-12-02 10:39
FYI There is also issue23034 where is proposed "-X showalloccount" option to suppress the output, we use (custom patch) environment variable to filter out the verbose output in Fedora.
msg255702 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-12-02 10:46
Yes, I also don't want to use 00141-fix-tests_with_COUNT_ALLOCS-v5.patch if there is better alternative. See issue23034 (I'm uncertain only in option name).
msg255703 - (view) Author: Bohuslav "Slavek" Kabrda (bkabrda) * Date: 2015-12-02 10:47
> IMHO these two debug features superseded COUNT_ALLOCS. Please try to convince me of the use case of this very old debug feature.

I no longer use this feature and I think that noone does. As you said, it seems to be obsoleted by other new features, so my vote would be to remove COUNT_ALLOCS altogether.
msg255711 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-12-02 12:00
I propose to emit a compiler warning (or even an error?) in 3.5.x and drop
the code in 3.6. I don't think that a long deprecation period is requied.
The feature is not widely used.
msg269772 - (view) Author: Roundup Robot (python-dev) Date: 2016-07-03 19:31
New changeset 5abf6cdcac4d by Serhiy Storchaka in branch '3.5':
Issue #19527: Fixed tests with defined COUNT_ALLOCS.
https://hg.python.org/cpython/rev/5abf6cdcac4d

New changeset e7d84ecdd37d by Serhiy Storchaka in branch 'default':
Issue #19527: Fixed tests with defined COUNT_ALLOCS.
https://hg.python.org/cpython/rev/e7d84ecdd37d
msg269791 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-04 16:33
All tests now are passed in 3.6 on Linux. Making them passing in 3.5 requires too much changes that are not needed in 3.6. I don't think we need to pollute tests with these temporary workarounds.
msg304492 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-10-17 09:25
New changeset 7b4ba62e388474e811268322b47f80d464933541 by Victor Stinner in branch '2.7':
[2.7] bpo-31692: Add PYTHONSHOWALLOCCOUNT env var (GH-3927)
https://github.com/python/cpython/commit/7b4ba62e388474e811268322b47f80d464933541
History
Date User Action Args
2017-10-17 09:25:27vstinnersetmessages: + msg304492
2017-10-09 09:21:52vstinnersetpull_requests: + pull_request3902
2017-10-06 19:58:09vstinnersetpull_requests: + pull_request3883
2016-07-04 16:33:17serhiy.storchakasetstatus: open -> closed
versions: + Python 3.6, - Python 3.4
messages: + msg269791

resolution: fixed
stage: patch review -> resolved
2016-07-03 19:31:16python-devsetnosy: + python-dev
messages: + msg269772
2015-12-02 12:00:03vstinnersetmessages: + msg255711
2015-12-02 10:47:19bkabrdasetmessages: + msg255703
2015-12-02 10:46:25serhiy.storchakasetmessages: + msg255702
2015-12-02 10:39:35rkuskasetmessages: + msg255701
2015-12-02 10:33:16vstinnersetmessages: + msg255700
2015-12-02 10:28:45vstinnersetmessages: + msg255698
2015-12-02 10:03:31rkuskasetnosy: + rkuska
messages: + msg255697
2014-12-13 17:47:36serhiy.storchakasetfiles: + 00141-fix-tests_with_COUNT_ALLOCS-v5.patch

dependencies: + Dynamically control debugging output
messages: + msg232615
2014-12-11 20:55:01serhiy.storchakasetfiles: + 00141-fix-tests_with_COUNT_ALLOCS-v4.patch

messages: + msg232492
2014-12-11 12:07:34serhiy.storchakasetassignee: serhiy.storchaka
2014-12-11 11:22:03bkabrdasetfiles: + 00141-fix-tests_with_COUNT_ALLOCS-v3.patch

messages: + msg232469
2014-12-10 22:40:04serhiy.storchakasetmessages: + msg232450
2014-12-10 22:35:46serhiy.storchakasetversions: + Python 3.5
nosy: + serhiy.storchaka

components: + Tests
type: behavior
stage: patch review
2014-01-10 10:33:30bkabrdasetmessages: + msg207848
2014-01-10 10:28:02vstinnersetmessages: + msg207845
2014-01-10 10:25:22bkabrdasetfiles: + 00141-fix-tests_with_COUNT_ALLOCS-v2.patch

messages: + msg207844
2013-11-08 12:37:47vstinnersetnosy: + vstinner
2013-11-08 12:30:50bkabrdasetfiles: + 00141-fix-tests_with_COUNT_ALLOCS.patch
keywords: + patch
messages: + msg202415
2013-11-08 12:30:13bkabrdacreate