Issue32542
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2018-01-12 18:56 by Michael.Felt, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Messages (6) | |||
---|---|---|---|
msg309869 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2018-01-12 18:56 | |
in issue25582 - the issue is not (yet) resolved. Perhaps this one can be closed and issue25582 reopened. Both from python2-2.7.14 and "git master" I am getting: michael@x071:[/data/prj/python/git/gcc-python3-3.7]./python -m unittest -v ctypes.test.test_pointers < test_abstract (ctypes.test.test_pointers.PointersTestCase) ... ok test_basic (ctypes.test.test_pointers.PointersTestCase) ... ok test_basics (ctypes.test.test_pointers.PointersTestCase) ... ok test_bug_1467852 (ctypes.test.test_pointers.PointersTestCase) ... ok test_c_void_p (ctypes.test.test_pointers.PointersTestCase) ... ok test_callbacks_with_pointers (ctypes.test.test_pointers.PointersTestCase) ... ok test_change_pointers (ctypes.test.test_pointers.PointersTestCase) ... ok test_charpp (ctypes.test.test_pointers.PointersTestCase) Test that a character pointer-to-pointer is correctly passed ... ok test_from_address (ctypes.test.test_pointers.PointersTestCase) ... ok test_other (ctypes.test.test_pointers.PointersTestCase) ... ok test_pass_pointers (ctypes.test.test_pointers.PointersTestCase) ... ok test_pointer_crash (ctypes.test.test_pointers.PointersTestCase) ... ok test_pointer_type_name (ctypes.test.test_pointers.PointersTestCase) ... ok test_pointer_type_str_name (ctypes.test.test_pointers.PointersTestCase) ... ERROR test_pointers_bool (ctypes.test.test_pointers.PointersTestCase) ... ok ====================================================================== ERROR: test_pointer_type_str_name (ctypes.test.test_pointers.PointersTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/prj/python/git/gcc-python3-3.7/Lib/ctypes/test/test_pointers.py", line 208, in test_pointer_type_str_name large_string = 'T' * 2 ** 25 MemoryError ---------------------------------------------------------------------- Ran 15 tests in 0.319s FAILED (errors=1) +++++ Looking at the test source: +196 def test_pointer_type_name(self): +197 LargeNamedType = type('T' * 2 ** 25, (Structure,), {}) +198 self.assertTrue(POINTER(LargeNamedType)) +199 +200 # to not leak references, we must clean _pointer_type_cache +201 from ctypes import _pointer_type_cache +202 del _pointer_type_cache[LargeNamedType] +203 +204 def test_pointer_type_str_name(self): +205 large_string = 'T' * 2 ** 25 +206 P = POINTER(large_string) +207 self.assertTrue(P) +208 +209 # to not leak references, we must clean _pointer_type_cache +210 from ctypes import _pointer_type_cache +211 del _pointer_type_cache[id(P)] **** After changing the exponent (** 25) to "** 23" on either line 197 OR 205 - ALL test succeed - After changing the exponent to " ** 24" on BOTH lines 197 and 205, all tests pass. **** My concern is that the "patch" from issue 25582 the "del _pointer_type_cache[]" statement is not freeing memory. +++++ What can I add to the test to debug! +++++ p.s. - results are the same on AIX (5.3 and 6.1) Python2-2.7.14 and Pyhton3-3.7.X (git master). Compiler does not seem to matter (both xlc and gcc). 32-bit, default memory model (256 MByte max - aka (2 ** 28) for data and malloc. I could also try changing the memory model - but will do that only if the test, by definition, is not "free()ing" the memory used, when done. HTH Michael |
|||
msg309874 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * | Date: 2018-01-12 22:48 | |
Issue25582 fixed a memory leak. If run the tests repeatedly every iteration leaked 100 MB of memory. If you get a MemoryError with a single iteration, this means that your machine just doesn't have enough memory for tests. Tests require around 500-600MB of memory on 32-bit platform. |
|||
msg309942 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2018-01-15 01:56 | |
Thanks for the clarification. Being curious, is there a way to see what the size of the cache is? I want to believe, but i do not have the impression memory is being reallocated to later users. My gut feeling is that the code change permits repeated calls to a function in a way that it reuses memory it received before (rather than a memory leak of repeatedly allocating a brand new buffer). What I expected, more accurately hoped for, is that cache was either freed and/or made available for reuse. Since I am on my phone atm I am less successful at finding documentation. I would be very grateful for a link to the relevant information and I’ll look further, time permitting. Thx again, Michael Sent from my iPhone > On 12 Jan 2018, at 23:48, Serhiy Storchaka <report@bugs.python.org> wrote: > > > Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment: > > Issue25582 fixed a memory leak. If run the tests repeatedly every iteration leaked 100 MB of memory. If you get a MemoryError with a single iteration, this means that your machine just doesn't have enough memory for tests. Tests require around 500-600MB of memory on 32-bit platform. > > ---------- > nosy: +serhiy.storchaka > > _______________________________________ > Python tracker <report@bugs.python.org> > <https://bugs.python.org/issue32542> > _______________________________________ > |
|||
msg309954 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * | Date: 2018-01-15 08:46 | |
> Being curious, is there a way to see what the size of the cache is? len(_pointer_type_cache)? |
|||
msg309955 - (view) | Author: STINNER Victor (vstinner) * | Date: 2018-01-15 09:06 | |
ctypes test cases tearDown() may always clear _pointer_type_cache. Would it help? |
|||
msg361295 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2020-02-03 14:19 | |
Not an issue in 3.9, so, closing: "not relevant" |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:58:56 | admin | set | github: 76723 |
2020-02-03 14:19:10 | Michael.Felt | set | status: open -> closed messages: + msg361295 stage: resolved |
2018-01-15 09:06:21 | vstinner | set | nosy:
+ vstinner messages: + msg309955 |
2018-01-15 08:46:04 | serhiy.storchaka | set | messages: + msg309954 |
2018-01-15 01:56:41 | Michael.Felt | set | messages: + msg309942 |
2018-01-12 22:48:32 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg309874 |
2018-01-12 18:56:34 | Michael.Felt | create |