Author maru-n
Recipients maru-n
Date 2019-06-07.11:53:38
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1559908419.41.0.390422965351.issue37193@roundup.psfhosted.org>
In-reply-to
Content
UDP/TCPServer with socketserver.ThreadingMixin class (also ThreadingTCPServer and ThreadingUDPServer class) seems to be memory leak while running the server.

https://docs.python.org/3/library/socketserver.html#socketserver.ThreadingMixIn

My code which wrote to check this is the following.

```
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
    def handle(self):
        data = str(self.request.recv(1024), 'ascii')
        cur_thread = threading.current_thread()
        response = bytes("{}: {}".format(cur_thread.name, data), 'ascii')
        self.request.sendall(response)


if __name__ == "__main__":
    HOST, PORT = "localhost", 0

    server = socketserver.ThreadingTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
    server.daemon_threads = False
    server.block_on_close = True

    with server:
        ip, port = server.server_address

        server_thread = threading.Thread(target=server.serve_forever)

        server_thread.daemon = True
        server_thread.start()

        for i in range(1000):
            with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
                sock.connect(server.server_address)
                sock.sendall(bytes("hello", 'ascii'))
                response = str(sock.recv(1024), 'ascii')
                print("Received: {}".format(response))
            time.sleep(0.01)

        server.server_close()
        server.shutdown()
```
( I wrote this based on https://docs.python.org/3/library/socketserver.html#asynchronous-mixins)

Then I checked memory usage with profiling tool.
(I used memory-profiler module https://pypi.org/project/memory-profiler/)

$ mprof run python mycode.py
$ mprof plot

I attached result plot.

And also I checked this also more long time and I found memory usage was increased endlessly.

My environment is

Hardware: MacBook Pro (15-inch, 2018)
OS: MacOS 10.14
Python 3.7.1


I guess it caused by a thread object is not released in spite of the thread finished to process request and thread object will be made infinitely until server_close() is called.
History
Date User Action Args
2019-06-07 11:53:39maru-nsetrecipients: + maru-n
2019-06-07 11:53:39maru-nsetmessageid: <1559908419.41.0.390422965351.issue37193@roundup.psfhosted.org>
2019-06-07 11:53:39maru-nlinkissue37193 messages
2019-06-07 11:53:38maru-ncreate