classification
Title: duplicate of memoryview from io.BufferedWriter leaks
Type: resource usage Stage: resolved
Components: IO Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: iritkatriel, jmadden, serhiy.storchaka
Priority: normal Keywords:

Created on 2018-11-21 15:12 by jmadden, last changed 2020-10-06 01:13 by methane. This issue is now closed.

Messages (2)
msg330206 - (view) Author: Jason Madden (jmadden) * Date: 2018-11-21 15:12
Using Python 2.7.15, if a BufferedWriter wraps an IO object that duplicates the memoryview passed to the IO object's `write` method, that memoryview leaks. 

This script demonstrates the problem by leaking a memoryview for each iteration of the loop (if the `flush` is skipped, the leaks are less frequent but still occur):

```
from __future__ import print_function

import io
import gc

def count_memoryview():
    result = 0
    for x in gc.get_objects():
        if type(x) is memoryview:
            result += 1
    return result

class FileLike(object):
    closed = False

    def writable(self):
        return True

    def write(self, data):
        memoryview(data) # XXX: This causes the problem
        return len(data)


bf = io.BufferedWriter(FileLike())

i = 0
memoryview_count = 0

while True:
    if i == 0 or i % 100 == 0:
        # This reports 100 new memoryview objects each time
        old = memoryview_count
        new = count_memoryview()
        print(i, "memoryview", new, "+%s" % (new - old))
        memoryview_count = new

    bf.write(b"test")
    bf.flush()
    i += 1
```

The leak can also be observed using the operating system's memory monitoring tools for the process (seen on both Fedora and macOS).

Commenting out the line in `FileLike.write` that makes a duplicate memoryview of the given buffer solves the leak. 

Deleting the BufferedWriter doesn't appear to reclaim the leaked memoryviews.

I can't duplicate this in Python 3.4 or above.

Originally reported to gevent in https://github.com/gevent/gevent/issues/1318

Possibly related to Issue 26720 and Issue 15994
msg378077 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-10-05 22:37
I tried the script on master (windows), and the problem is not reproducible.


The OP also wrote that this is a 2.7-only issue, so it seems to be out of date.
History
Date User Action Args
2020-10-06 01:13:07methanesetstatus: open -> closed
resolution: out of date
stage: resolved
2020-10-05 22:37:37iritkatrielsetnosy: + iritkatriel, serhiy.storchaka
type: resource usage
messages: + msg378077
2018-11-21 15:12:32jmaddencreate