classification
Title: python.exe crashes - lzma?
Type: crash Stage:
Components: Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jonathan-lp
Priority: normal Keywords:

Created on 2018-11-23 14:21 by jonathan-lp, last changed 2018-11-24 09:35 by jonathan-lp.

Files
File name Uploaded Description Edit
python-mini.7z jonathan-lp, 2018-11-23 14:21 mini dump of process at crash-time. Created with Process Explorer
Messages (2)
msg330327 - (view) Author: Jonathan (jonathan-lp) Date: 2018-11-23 14:21
Python 3.6.3 on Windows 7 64 bit.

I keep getting intermittent crashes of Python.exe with my project. The error is below and as best I can see seems to be indicating the issue with the LZMA module.

All I'm doing is using it to compress web-pages (typically XML ones). Problem is, I can't replicate it. When I try and run the process again, the page that previously crashed it now works. I've had this happen three times so far (out of about 150,000 pages downloaded and compressed).

My use of LZMA is quite simply:

from lzma import compress
compress(page_content, preset=5)

The pages it has crashed on so far (but which work fine now):
http://www.geoportal.rlp.de/mapbender/php/wms.php?inspire=1&layer_id=35833&request=GetCapabilities&service=WMS&version=1.3.0
https://maps.runnymede.gov.uk/geoserver/planning/lp_ancient_woodland/wms?request=GetCapabilities&service=WMS&version=1.3.0
http://serviziogc.regione.fvg.it/geoserver/GEOLOGIA/wms?request=GetCapabilities&service=WMS&version=1.3.0

I took a mini dump (attached) and full dump using Process Explorer. The full dump is 100MB compressed (1GB uncompressed).


----

Problem signature:
  Problem Event Name:	APPCRASH
  Application Name:	python.exe
  Application Version:	3.6.3150.1013
  Application Timestamp:	59d3d3a3
  Fault Module Name:	_lzma.pyd
  Fault Module Version:	3.6.3150.1013
  Fault Module Timestamp:	59d3d343
  Exception Code:	c0000005
  Exception Offset:	000000000002e368
  OS Version:	6.1.7601.2.1.0.768.3
  Locale ID:	2057
  Additional Information 1:	d00b
  Additional Information 2:	d00be1cd6ce69f4b081e66c649a14f90
  Additional Information 3:	8c17
  Additional Information 4:	8c17b9c4b6a39b7e31bf71b3b1374f1b
msg330373 - (view) Author: Jonathan (jonathan-lp) Date: 2018-11-24 09:35
In total my python.exe process crashed 6 times with the exact same error, compressing about 60,000 pages, so about 1 in 10,000 failed.
However, I have ran the same process monthly for the last several months and it has never done that before despite processing mostly the same sites and compressing similar quantities of pages. The code is completely unchanged from last month.

Upon reflection, the only thing that has changed since then is that I am now running BOINC (https://boinc.berkeley.edu/) in the background on the machine. So I have just re-ran the entire process with BOINC suspended. Not a single crash!
It seems that somehow BOINC and Python are interacting to cause Python to crash.
History
Date User Action Args
2018-11-24 09:35:12jonathan-lpsetmessages: + msg330373
2018-11-23 14:21:13jonathan-lpcreate