classification
Title: Reference leaks on Windows
Type: resource usage Stage: resolved
Components: Windows Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: paul.moore, serhiy.storchaka, steve.dower, tim.golden, vstinner, xiang.zhang, zach.ware
Priority: high Keywords:

Created on 2016-12-21 22:28 by zach.ware, last changed 2017-10-02 09:31 by vstinner. This issue is now closed.

Messages (4)
msg283792 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-12-21 22:28
Discussion in #29034 inspired me to run a refleak check on Windows, with the following results from a 32-bit build of the current 3.6 branch:

C:\cpython\3.6>PCbuild\build.bat -e -d
C:\cpython\3.6>python -m test -uall -R 3:3:refleak_log.txt --timeout=7200

...

24 tests failed:
    test_atexit test_capi test_concurrent_futures test_fileio
    test_functools test_glob test_idle test_io test_math
    test_multiprocessing_spawn test_ntpath test_os test_platform
    test_posixpath test_shutil test_ssl test_sys test_tarfile
    test_tempfile test_threading test_unicode_file_functions
    test_unittest test_warnings test_winconsoleio

1 test altered the execution environment:
    test_distutils

28 tests skipped:
    test_crypt test_curses test_dbm_gnu test_dbm_ndbm test_devpoll
    test_epoll test_fcntl test_fork1 test_gdb test_grp test_ioctl
    test_kqueue test_nis test_openpty test_ossaudiodev test_pipes
    test_poll test_posix test_pty test_pwd test_readline test_resource
    test_spwd test_syslog test_threadsignals test_wait3 test_wait4
    test_zipfile64

Total duration: 190 min 60 sec
Tests result: FAILURE

test_atexit leaked [12, 12, 12] references, sum=36
test_atexit leaked [4, 6, 6] memory blocks, sum=16
test_capi leaked [6, 6, 6] references, sum=18
test_capi leaked [2, 4, 4] memory blocks, sum=10
test_concurrent_futures leaked [792, 783, 765] references, sum=2340
test_concurrent_futures leaked [264, 263, 257] memory blocks, sum=784
test_fileio leaked [1, 2, 2] memory blocks, sum=5
test_functools leaked [0, 3, 2] memory blocks, sum=5
test_glob leaked [265, 266, 266] memory blocks, sum=797
test_idle leaked [471, 471, 471] references, sum=1413
test_idle leaked [209, 211, 211] memory blocks, sum=631
test_io leaked [2, 3, 3] memory blocks, sum=8
test_multiprocessing_spawn leaked [168, 117, 144] references, sum=429
test_multiprocessing_spawn leaked [55, 43, 50] memory blocks, sum=148
test_ntpath leaked [103, 104, 104] memory blocks, sum=311
test_os leaked [33, 33, 33] references, sum=99
test_os leaked [58, 60, 60] memory blocks, sum=178
test_platform leaked [12, 12, 12] references, sum=36
test_platform leaked [4, 6, 6] memory blocks, sum=16
test_posixpath leaked [3, 4, 4] memory blocks, sum=11
test_shutil leaked [5, 6, 6] memory blocks, sum=17
test_ssl leaked [3, 4, 4] memory blocks, sum=11
test_sys leaked [9, 9, 9] references, sum=27
test_sys leaked [3, 5, 5] memory blocks, sum=13
test_tarfile leaked [4, 5, 5] memory blocks, sum=14
test_tempfile leaked [82, 83, 83] memory blocks, sum=248
test_threading leaked [12, 12, 12] references, sum=36
test_threading leaked [4, 6, 6] memory blocks, sum=16
test_unicode_file_functions leaked [5, 6, 6] memory blocks, sum=17
test_winconsoleio leaked [38, 38, 38] references, sum=114
test_winconsoleio leaked [0, 1, 2] memory blocks, sum=3
msg291802 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-04-17 16:39
Here's an updated refleak run, on master as of last Thursday:

test_atexit leaked [12, 12, 12] references, sum=36
test_atexit leaked [4, 6, 6] memory blocks, sum=16
test_capi leaked [6, 6, 6] references, sum=18
test_capi leaked [2, 4, 4] memory blocks, sum=10
test_asyncio leaked [4, 1, 1] memory blocks, sum=6
test_concurrent_futures leaked [822, 807, 810] references, sum=2439
test_concurrent_futures leaked [274, 271, 272] memory blocks, sum=817
test_import leaked [-2, 2, 0] references, sum=0
test_import leaked [-2, 4, 2] memory blocks, sum=4
test_logging leaked [-21, 1, 1] memory blocks, sum=-19
test_os leaked [6, 7, 7] memory blocks, sum=20
test_platform leaked [12, 12, 12] references, sum=36
test_platform leaked [4, 6, 6] memory blocks, sum=16
test_sys leaked [9, 9, 9] references, sum=27
test_sys leaked [3, 5, 5] memory blocks, sum=13
test_threading leaked [12, 12, 12] references, sum=36
test_threading leaked [4, 6, 6] memory blocks, sum=16
test_xml_etree_c leaked [6, -6, 0] references, sum=0
test_xml_etree_c leaked [1, 1, 2] memory blocks, sum=4
test_multiprocessing_spawn leaked [159, 180, 162] references, sum=501
test_multiprocessing_spawn leaked [54, 62, 55] memory blocks, sum=171

I ran this one with `-j0` which may have affected the results slightly, but shaved 40 minutes off the run time.
msg303446 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-10-01 02:17
Victor has cleaned these up, and we now have a buildbot running refleak checks on Windows every day (as long as it's not hung :)
msg303517 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-10-02 09:31
> Victor has cleaned these up, and we now have a buildbot running refleak checks on Windows every day (as long as it's not hung :)

\o/ Hopefully I was alone ;-) https://haypo.github.io/contrib-cpython-2017q2-part2.html
History
Date User Action Args
2017-10-02 09:31:29vstinnersetmessages: + msg303517
2017-10-01 02:17:21zach.waresetstatus: open -> closed
resolution: fixed
messages: + msg303446

stage: needs patch -> resolved
2017-04-17 16:39:42zach.waresetmessages: + msg291802
2016-12-21 22:28:41zach.warecreate