classification
Title: multiprocessing tests log: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
Type: Stage: resolved
Components: Tests Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: lukasz.langa, miss-islington, sobolevn, vstinner
Priority: normal Keywords: patch

Created on 2021-09-15 13:29 by vstinner, last changed 2021-09-21 18:37 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28377 merged sobolevn, 2021-09-15 20:33
PR 28499 merged miss-islington, 2021-09-21 17:56
PR 28500 merged miss-islington, 2021-09-21 17:57
Messages (7)
msg401837 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-15 13:29
0:03:25 load avg: 12.33 [250/427] test_multiprocessing_forkserver passed (...)
/home/vstinner/python/main/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/home/vstinner/python/main/Lib/multiprocessing/resource_tracker.py:237: UserWarning: resource_tracker: '/psm_33506d9a': [Errno 2] No such file or directory: '/psm_33506d9a'
  warnings.warn('resource_tracker: %r: %s' % (name, e))
msg401866 - (view) Author: Nikita Sobolev (sobolevn) * Date: 2021-09-15 17:47
I would like to work on this, if no one has started yet.
msg401902 - (view) Author: Nikita Sobolev (sobolevn) * Date: 2021-09-15 20:31
The problem was in `test_shared_memory_cleaned_after_process_termination` test.

```
» ./python.exe -m test -v test_multiprocessing_forkserver -m test_shared_memory_cleaned_after_process_termination
== CPython 3.11.0a0 (heads/main:09b4ad11f3, Sep 15 2021, 20:50:50) [Clang 11.0.0 (clang-1100.0.33.16)]
== macOS-10.14.6-x86_64-i386-64bit little-endian
== cwd: /Users/sobolev/Desktop/cpython/build/test_python_14874æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.98 Run tests sequentially
0:00:00 load avg: 1.98 [1/1] test_multiprocessing_forkserver
test_shared_memory_cleaned_after_process_termination (test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory) ... ok
/Users/sobolev/Desktop/cpython/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/Users/sobolev/Desktop/cpython/Lib/multiprocessing/resource_tracker.py:237: UserWarning: resource_tracker: '/psm_67480e45': [Errno 2] No such file or directory: '/psm_67480e45'
  warnings.warn('resource_tracker: %r: %s' % (name, e))

----------------------------------------------------------------------
Ran 1 test in 0.775s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1.6 sec
Tests result: SUCCESS
```

My patch (location https://github.com/python/cpython/blob/51056b40e711d84692d099ac8970077b33c7fafd/Lib/test/_test_multiprocessing.py#L4179):

```
resource_tracker.unregister(f"/{name}", "shared_memory")
```

With this change no warning is generated:

```
» ./python.exe -m test -v test_multiprocessing_forkserver -m test_shared_memory_cleaned_after_process_termination
== CPython 3.11.0a0 (heads/main:09b4ad11f3, Sep 15 2021, 20:50:50) [Clang 11.0.0 (clang-1100.0.33.16)]
== macOS-10.14.6-x86_64-i386-64bit little-endian
== cwd: /Users/sobolev/Desktop/cpython/build/test_python_24281æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.80 Run tests sequentially
0:00:00 load avg: 1.80 [1/1] test_multiprocessing_forkserver
test_shared_memory_cleaned_after_process_termination (test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.732s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1.5 sec
Tests result: SUCCESS
```

My other highly-related PR where I refactor several `SharedMemory` tests: https://github.com/python/cpython/pull/28294
msg402336 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-21 17:49
New changeset f604cf1c377a7648e0686044e6e49900bfc9feef by Nikita Sobolev in branch 'main':
bpo-45209: fix `UserWarning: resource_tracker` in test_multiprocessing (GH-28377)
https://github.com/python/cpython/commit/f604cf1c377a7648e0686044e6e49900bfc9feef
msg402341 - (view) Author: miss-islington (miss-islington) Date: 2021-09-21 18:19
New changeset 09e5016f8bb07beb7d780df221febac2d2867ad0 by Miss Islington (bot) in branch '3.10':
bpo-45209: fix `UserWarning: resource_tracker` in test_multiprocessing (GH-28377)
https://github.com/python/cpython/commit/09e5016f8bb07beb7d780df221febac2d2867ad0
msg402342 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-21 18:37
New changeset 65268ab849173e9849b7fc94c3d73900b5410444 by Miss Islington (bot) in branch '3.9':
bpo-45209: fix `UserWarning: resource_tracker` in test_multiprocessing (GH-28377) (GH-28500)
https://github.com/python/cpython/commit/65268ab849173e9849b7fc94c3d73900b5410444
msg402343 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-21 18:37
Thanks Nikita Sobolev for the fix.
History
Date User Action Args
2021-09-21 18:37:39vstinnersetstatus: open -> closed
versions: + Python 3.9, Python 3.10
messages: + msg402343

resolution: fixed
stage: patch review -> resolved
2021-09-21 18:37:11vstinnersetmessages: + msg402342
2021-09-21 18:19:56miss-islingtonsetmessages: + msg402341
2021-09-21 17:57:48miss-islingtonsetpull_requests: + pull_request26895
2021-09-21 17:56:02miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26894
2021-09-21 17:49:49lukasz.langasetnosy: + lukasz.langa
messages: + msg402336
2021-09-15 20:33:50sobolevnsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26792
2021-09-15 20:31:55sobolevnsetmessages: + msg401902
2021-09-15 17:47:02sobolevnsetnosy: + sobolevn
messages: + msg401866
2021-09-15 13:29:13vstinnercreate