classification
Title: zlib.compressobj took too much memory on window
Type: resource usage Stage:
Components: Windows Versions: Python 3.7, Python 3.6, Python 3.3, Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: dennisding, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2017-06-29 05:56 by dennisding, last changed 2017-06-29 05:56 by dennisding.

Messages (1)
msg297257 - (view) Author: dennis (dennisding) Date: 2017-06-29 05:56
I'm working on a MMORPG Game server, and using the 32bit window Python(Python2.7.9 and Python3.6.0).
And i create a zlib.compressobj for every connection.

when the connection reach about 7500, the program raise an MemoryError exception. The Program only took me 700MB in memory, But the committed memory is about 2GB.
Here is my test result and test code:
-----------test result------------------
7613 compressobj created!
('memory info:', pmem(rss=736342016, vms=2057256960, num_page_faults=179812, pea
k_wset=736342016, wset=736342016, peak_paged_pool=130760, paged_pool=130584, pea
k_nonpaged_pool=28344, nonpaged_pool=28344, pagefile=2057256960, peak_pagefile=2
057256960, private=2057256960))

--------------test code ----------------
# -*- encoding:utf-8 -*-

import os
import psutil
import zlib

def test():
	result = []
	for i in range(10000):
		try:
			result.append(zlib.compressobj())
		except:
			print('%d compressobj created!'%(i))
			pid = os.getpid()
			p = psutil.Process(pid)
			print('memory info:', p.memory_info())
			break


if __name__ == '__main__':
	test()
History
Date User Action Args
2017-06-29 05:56:39dennisdingcreate