Title: email.parser.Parser is inefficient with large strings
Type: performance Stage: resolved
Components: Library (Lib) Versions: Python 2.6
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: r.david.murray Nosy List: marcio, r.david.murray
Priority: normal Keywords:

Created on 2010-02-24 09:44 by marcio, last changed 2010-12-28 04:24 by r.david.murray. This issue is now closed.

File name Uploaded Description Edit marcio, 2010-02-24 09:44 Simple speed test
Messages (2)
msg100019 - (view) Author: Márcio (marcio) Date: 2010-02-24 09:44
The email parser class is slow and memory intensive when dealing with sufficiently large strings.

For example, on a Windows 7 64-bit running at 1.60 GHz the attached test file gives the following results (number of seconds it took to parse a 10 MiB string):
Original: 76.6973627829
Modified: 0.231140741387
msg124760 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-28 04:24
Parser is a legacy API, and message_from_string (which uses it) is just a convenience function.  If performance is an issue for your application, call feedparser directly and optimize the feeding to suit your application.
Date User Action Args
2010-12-28 04:24:37r.david.murraysetstatus: open -> closed

messages: + msg124760
resolution: wont fix
stage: resolved
2010-08-01 22:56:11georg.brandlsetassignee: r.david.murray

nosy: + r.david.murray
2010-02-24 09:44:03marciocreate