Infinite recursion tests triggering a segfault #62275

brettcannon · 2013-05-28T01:02:08Z

BPO	18075
Nosy	@brettcannon, @ronaldoussoren, @vstinner, @larryhastings, @ned-deily, @miss-islington
PRs	bpo-34602: Avoid failures setting macOS stack resource limit #13011 [3.7] bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011) #13013 [3.6] bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011) #13014 bpo-34602: Avoid failures setting macOS stack resource limit #14546 [3.8] bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546) #14547 [3.7] bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546) #14548 [3.6] bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546) #14549
Files	issue-18075-osx-stacksize.txt issue-18075-osx-stacksize-update.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ronaldoussoren'
closed_at = <Date 2013-05-30.19:31:46.106>
created_at = <Date 2013-05-28.01:02:08.341>
labels = ['interpreter-core', 'type-crash', 'release-blocker']
title = 'Infinite recursion tests triggering a segfault'
updated_at = <Date 2019-07-02.22:34:03.946>
user = 'https://github.com/brettcannon'

bugs.python.org fields:

activity = <Date 2019-07-02.22:34:03.946>
actor = 'ned.deily'
assignee = 'ronaldoussoren'
closed = True
closed_date = <Date 2013-05-30.19:31:46.106>
closer = 'lukasz.langa'
components = ['Interpreter Core']
creation = <Date 2013-05-28.01:02:08.341>
creator = 'brett.cannon'
dependencies = []
files = ['30397', '30399']
hgrepos = []
issue_num = 18075
keywords = []
message_count = 18.0
messages = ['190162', '190192', '190200', '190205', '190206', '190354', '190355', '190356', '190358', '190378', '341111', '341113', '341117', '347112', '347114', '347116', '347118', '347166']
nosy_count = 7.0
nosy_names = ['brett.cannon', 'ronaldoussoren', 'vstinner', 'larry', 'ned.deily', 'python-dev', 'miss-islington']
pr_nums = ['13011', '13013', '13014', '14546', '14547', '14548', '14549']
priority = 'release blocker'
resolution = 'fixed'
stage = 'patch review'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue18075'
versions = ['Python 3.4']

brettcannon · 2013-05-28T01:02:07Z

If you run any test that has infinite recursion (test_json test_exceptions test_sys test_runpy) it will segfault with a fresh checkout under OS X 10.8.3 using Clang. Not sure how widespread this is.

I did check that I am using a clean checkout of the default branch.

ronaldoussoren · 2013-05-28T10:23:38Z

I can confirm the problem. It appears to be a stack overflow, when I increase the stack size of the main thead (by adding "-Wl,-stack_size,2faf000" to the link command for BUILDPYTHON) the crash in test_json goes away.

Appearently the default maximum stack size isn't large enough for the default value of the recursion limit.

An easy workaround (fix?) would be to add -Wl,-stack_size,VALUE to the link flags on OSX, for some sane value of VALUE. The value is the maximum size of the stack of the main thread and doesn't affect process size unless a process actually uses more stack space. It does affect the available free address space, and hence the maximum stack size shouldn't be increased too far (especially for 32-bit builds)

ronaldoussoren · 2013-05-28T12:31:27Z

The attached patch fixes the problem on OSX by increasing the maximum stack size of the main thread from 8M (the default) to 16M.

NOTE: The -Wl,-stack_size,... option cannot be added to LDFLAGS, ld errors out when that option is used when linking a shared library (such as the extensions or libpython.dylib)

ronaldoussoren · 2013-05-28T13:20:38Z

The update fixes the name error mention in rietveld.

brettcannon · 2013-05-28T13:26:37Z

LGTM

vstinner · 2013-05-30T11:49:08Z

Appearently the default maximum stack size isn't large enough for the default value of the recursion limit.

Why not changing the recursion limit instead of the size of the stack?

ronaldoussoren · 2013-05-30T12:18:36Z

I'd prefer to keep the default recursion limit the same as on Linux, changing the recursion limit on OSX would just introduce an unnecessary difference between the two platforms.

The patch changes the maximum stack size from 8 to 16 MByte, neither of which is huge and shouldn't cause other problems.

I'd also like to increase the default stack size for newly created threads (see bpo-18049) and will update that patch to create a 16 MByte stack as well.

vstinner · 2013-05-30T12:50:20Z

The patch changes the maximum stack size from 8 to 16 MByte, neither of which is huge and shouldn't cause other problems.

On Mac OS X: Is the memory allocated at Python startup, or on demand,
as the stack grows? If I am correct, the physical memory is allocated
on demand on Linux.

2013/5/30 Ronald Oussoren <report@bugs.python.org>:

Ronald Oussoren added the comment:

I'd prefer to keep the default recursion limit the same as on Linux, changing the recursion limit on OSX would just introduce an unnecessary difference between the two platforms.

The patch changes the maximum stack size from 8 to 16 MByte, neither of which is huge and shouldn't cause other problems.

I'd also like to increase the default stack size for newly created threads (see bpo-18049) and will update that patch to create a 16 MByte stack as well.

----------

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue18075\>

ronaldoussoren · 2013-05-30T13:33:40Z

On 30 May, 2013, at 14:50, STINNER Victor <report@bugs.python.org> wrote:

STINNER Victor added the comment:

> The patch changes the maximum stack size from 8 to 16 MByte, neither of which is huge and shouldn't cause other problems.

On Mac OS X: Is the memory allocated at Python startup, or on demand,
as the stack grows? If I am correct, the physical memory is allocated
on demand on Linux.

Memory for the stack is allocated on demand, the parameter sets the maximum size that the stack can grow to. See also man ld(1).

Ronald

python-dev · 2013-05-30T19:02:29Z

New changeset b07ad4b5e349 by Łukasz Langa in branch 'default':
Fixed bpo-18075 - Infinite recursion tests triggering a segfault on Mac OS X
http://hg.python.org/cpython/rev/b07ad4b5e349

ned-deily · 2019-04-29T19:07:44Z

New changeset 883dfc6 by Ned Deily in branch 'master':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011)
883dfc6

miss-islington · 2019-04-29T19:27:39Z

New changeset 52a5b71 by Miss Islington (bot) in branch '3.7':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011)
52a5b71

ned-deily · 2019-04-29T19:57:20Z

New changeset fbe2a13 by Ned Deily (Miss Islington (bot)) in branch '3.6':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011) (GH-13014)
fbe2a13

ned-deily · 2019-07-02T07:12:37Z

New changeset 5bbbc73 by Ned Deily in branch 'master':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
5bbbc73

miss-islington · 2019-07-02T07:31:13Z

New changeset bd92b94 by Miss Islington (bot) in branch '3.8':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
bd92b94

miss-islington · 2019-07-02T07:38:42Z

New changeset bf82cd3 by Miss Islington (bot) in branch '3.7':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
bf82cd3

ned-deily · 2019-07-02T07:49:02Z

New changeset 782854f by Ned Deily (Miss Islington (bot)) in branch '3.6':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546) (GH-14549)
782854f

ned-deily · 2019-07-02T22:34:04Z

New changeset dcc0eb3 by Ned Deily (Miss Islington (bot)) in branch '3.7':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
dcc0eb3

brettcannon added release-blocker interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump labels May 28, 2013

brettcannon assigned ronaldoussoren May 28, 2013

ambv closed this as completed May 30, 2013

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Infinite recursion tests triggering a segfault #62275

Infinite recursion tests triggering a segfault #62275

brettcannon commented May 28, 2013

brettcannon commented May 28, 2013

ronaldoussoren commented May 28, 2013

ronaldoussoren commented May 28, 2013

ronaldoussoren commented May 28, 2013

brettcannon commented May 28, 2013

vstinner commented May 30, 2013

ronaldoussoren commented May 30, 2013

vstinner commented May 30, 2013

ronaldoussoren commented May 30, 2013

python-dev mannequin commented May 30, 2013

ned-deily commented Apr 29, 2019

miss-islington commented Apr 29, 2019

ned-deily commented Apr 29, 2019

ned-deily commented Jul 2, 2019

miss-islington commented Jul 2, 2019

miss-islington commented Jul 2, 2019

ned-deily commented Jul 2, 2019

ned-deily commented Jul 2, 2019

Infinite recursion tests triggering a segfault #62275

Infinite recursion tests triggering a segfault #62275

Comments

brettcannon commented May 28, 2013

brettcannon commented May 28, 2013

ronaldoussoren commented May 28, 2013

ronaldoussoren commented May 28, 2013

ronaldoussoren commented May 28, 2013

brettcannon commented May 28, 2013

vstinner commented May 30, 2013

ronaldoussoren commented May 30, 2013

vstinner commented May 30, 2013

ronaldoussoren commented May 30, 2013

python-dev mannequin commented May 30, 2013

ned-deily commented Apr 29, 2019

miss-islington commented Apr 29, 2019

ned-deily commented Apr 29, 2019

ned-deily commented Jul 2, 2019

miss-islington commented Jul 2, 2019

miss-islington commented Jul 2, 2019

ned-deily commented Jul 2, 2019

ned-deily commented Jul 2, 2019