classification
Title: Memory Leak - ThreadingUDPServer
Type: performance Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Memory leak while running TCP/UDPServer with socketserver.ThreadingMixIn
View: 37193
Assigned To: Nosy List: danf@dataforge.on.ca, vstinner
Priority: normal Keywords:

Created on 2019-06-24 14:53 by danf@dataforge.on.ca, last changed 2019-06-24 15:01 by vstinner. This issue is now closed.

Messages (3)
msg346410 - (view) Author: Daniel W Forsyth (danf@dataforge.on.ca) Date: 2019-06-24 14:53
After putting a basic ThreadingUDPServer under load (500 messages per/second)  I noticed that after a night it was consuming a lot of RAM given it does nothing with the data.

On inception, I noticed the _thread count inside the server was growing forever even though the sub-threads are done.

Setup a basic ThreadingUDPSever with handler that does nothing and check the request_queue_size, it seems to grow without limit.
msg346411 - (view) Author: Daniel W Forsyth (danf@dataforge.on.ca) Date: 2019-06-24 14:59
The only way I could figure out to control it was to do this in a thread;

        for thread in server._threads:  # type: Thread
            if not thread.is_alive():
                server._threads.remove(thread)

Shouldn't the server process do this when the thread is done?
msg346412 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-24 15:01
This issue is a duplicate of bpo-37193.
History
Date User Action Args
2019-06-24 15:01:07vstinnersetstatus: open -> closed

superseder: Memory leak while running TCP/UDPServer with socketserver.ThreadingMixIn

nosy: + vstinner
messages: + msg346412
resolution: duplicate
stage: resolved
2019-06-24 14:59:58danf@dataforge.on.casetmessages: + msg346411
2019-06-24 14:53:55danf@dataforge.on.cacreate