classification
Title: test_bad_getattr crashes on APPX test
Type: crash Stage: commit review
Components: Windows Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: lukasz.langa, miss-islington, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2020-04-30 23:10 by steve.dower, last changed 2020-05-05 18:03 by miss-islington.

Pull Requests
URL Status Linked Edit
PR 19845 merged steve.dower, 2020-05-01 20:00
PR 19941 merged miss-islington, 2020-05-05 17:46
Messages (8)
msg367803 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-04-30 23:10
The Windows CI machines on Azure Pipelines run additional tests to check an "installed" layout and with the UWP entry point that's used for the Windows Store package.

These tests have been failing intermittently (though regularly) with a stack overflow crash in the PyPickler tests.

Example: https://dev.azure.com/Python/cpython/_build/results?buildId=62055&view=results

test_attribute_name_interning (test.test_pickle.PyPicklerTests) ... ok
  File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
  File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
  File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
  File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
...

I assume this is due to having more code on the start at the start, and so the recursion limit isn't low enough. But it might also be worth checking this particular case to see whether there is unnecessary data being kept on the stack (e.g. in local C variables).

The crash occurs in both 3.8 and master, but not 3.7.
msg367880 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-01 20:01
Turns out the stack reservation was different, which is why it was crashing.

As part of diagnosing it, I added the recursion count to faulthandler's output on Windows, but couldn't see if/where to do it for other platforms - Victor, any suggestions?
msg367881 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-01 20:02
Also, it was really test_bad_getattr that was crashing.
msg367888 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-01 21:26
> Also, it was really test_bad_getattr that was crashing.

Ah, an old friend. It seems like an issue with the maximum stack size. Here are my notes:
https://pythondev.readthedocs.io/unstable_tests.html#unlimited-recursion

Either reduce Python maximum stack depth, or increase the C stack size.
msg368060 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-04 15:50
Yeah, I already got that part. If you check the PR, I added some better diagnostics to faulthandler for this case, but I don't see where I can add it for other platforms?
msg368185 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-05 17:45
New changeset ac4bf424119d1300f57929120968e216a85d3a25 by Steve Dower in branch 'master':
bpo-40458: Increase reserved stack space to prevent overflow crash on Windows (GH-19845)
https://github.com/python/cpython/commit/ac4bf424119d1300f57929120968e216a85d3a25
msg368186 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-05 17:48
Merged without the improved diagnostics. I might add that later if I get time.

Ɓukasz - this is a fairly trivial crash fix that'd be nice (and safe) to pull into 3.8.3, but not critical. Your call.
msg368189 - (view) Author: miss-islington (miss-islington) Date: 2020-05-05 18:03
New changeset a6a116c1b964b3d1fdff0f533861ed2a2227de1f by Miss Islington (bot) in branch '3.8':
bpo-40458: Increase reserved stack space to prevent overflow crash on Windows (GH-19845)
https://github.com/python/cpython/commit/a6a116c1b964b3d1fdff0f533861ed2a2227de1f
History
Date User Action Args
2020-05-05 18:03:24miss-islingtonsetmessages: + msg368189
2020-05-05 17:48:12steve.dowersetnosy: + lukasz.langa
messages: + msg368186

assignee: steve.dower
stage: patch review -> commit review
2020-05-05 17:46:01miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request19255
2020-05-05 17:45:42steve.dowersetmessages: + msg368185
2020-05-04 15:50:03steve.dowersetmessages: + msg368060
2020-05-01 21:26:47vstinnersetmessages: + msg367888
2020-05-01 20:02:04steve.dowersetmessages: + msg367881
title: test_attribute_name_interning crashes on APPX test -> test_bad_getattr crashes on APPX test
2020-05-01 20:01:39steve.dowersetnosy: + vstinner
type: crash
messages: + msg367880
2020-05-01 20:00:06steve.dowersetkeywords: + patch
stage: patch review
pull_requests: + pull_request19163
2020-04-30 23:10:27steve.dowercreate