classification
Title: python.exe crashes - lzma?
Type: crash Stage: resolved
Components: Versions: Python 3.6
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: iritkatriel, jonathan-lp
Priority: normal Keywords:

Created on 2018-11-23 14:21 by jonathan-lp, last changed 2021-09-09 23:07 by iritkatriel. This issue is now closed.

Files
File name Uploaded Description Edit
python-mini.7z jonathan-lp, 2018-11-23 14:21 mini dump of process at crash-time. Created with Process Explorer
Messages (3)
msg330327 - (view) Author: Jonathan (jonathan-lp) Date: 2018-11-23 14:21
Python 3.6.3 on Windows 7 64 bit.

I keep getting intermittent crashes of Python.exe with my project. The error is below and as best I can see seems to be indicating the issue with the LZMA module.

All I'm doing is using it to compress web-pages (typically XML ones). Problem is, I can't replicate it. When I try and run the process again, the page that previously crashed it now works. I've had this happen three times so far (out of about 150,000 pages downloaded and compressed).

My use of LZMA is quite simply:

from lzma import compress
compress(page_content, preset=5)

The pages it has crashed on so far (but which work fine now):
http://www.geoportal.rlp.de/mapbender/php/wms.php?inspire=1&layer_id=35833&request=GetCapabilities&service=WMS&version=1.3.0
https://maps.runnymede.gov.uk/geoserver/planning/lp_ancient_woodland/wms?request=GetCapabilities&service=WMS&version=1.3.0
http://serviziogc.regione.fvg.it/geoserver/GEOLOGIA/wms?request=GetCapabilities&service=WMS&version=1.3.0

I took a mini dump (attached) and full dump using Process Explorer. The full dump is 100MB compressed (1GB uncompressed).


----

Problem signature:
  Problem Event Name:	APPCRASH
  Application Name:	python.exe
  Application Version:	3.6.3150.1013
  Application Timestamp:	59d3d3a3
  Fault Module Name:	_lzma.pyd
  Fault Module Version:	3.6.3150.1013
  Fault Module Timestamp:	59d3d343
  Exception Code:	c0000005
  Exception Offset:	000000000002e368
  OS Version:	6.1.7601.2.1.0.768.3
  Locale ID:	2057
  Additional Information 1:	d00b
  Additional Information 2:	d00be1cd6ce69f4b081e66c649a14f90
  Additional Information 3:	8c17
  Additional Information 4:	8c17b9c4b6a39b7e31bf71b3b1374f1b
msg330373 - (view) Author: Jonathan (jonathan-lp) Date: 2018-11-24 09:35
In total my python.exe process crashed 6 times with the exact same error, compressing about 60,000 pages, so about 1 in 10,000 failed.
However, I have ran the same process monthly for the last several months and it has never done that before despite processing mostly the same sites and compressing similar quantities of pages. The code is completely unchanged from last month.

Upon reflection, the only thing that has changed since then is that I am now running BOINC (https://boinc.berkeley.edu/) in the background on the machine. So I have just re-ran the entire process with BOINC suspended. Not a single crash!
It seems that somehow BOINC and Python are interacting to cause Python to crash.
msg401530 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-09-09 23:07
This sounds like it may very well be memory corruption by BOINC.  In any case, Python 3.6.3 is no longer maintained.
History
Date User Action Args
2021-09-09 23:07:43iritkatrielsetstatus: open -> closed

nosy: + iritkatriel
messages: + msg401530

resolution: third party
stage: resolved
2018-11-24 09:35:12jonathan-lpsetmessages: + msg330373
2018-11-23 14:21:13jonathan-lpcreate