S:\src\svn\svn.python.org\projects\python\trunk.x64\PCbuild>amd64\python_d
..\lib\test\test_marshal.py
test_bool (main.IntTestCase) ... ok
test_int64 (main.IntTestCase) ... ok
test_ints (main.IntTestCase) ... ok
test_floats (main.FloatTestCase) ... ok
test_buffer (main.StringTestCase) ... ok
test_string (main.StringTestCase) ... ok
test_unicode (main.StringTestCase) ... ok
test_code (main.CodeTestCase) ... ok
test_dict (main.ContainerTestCase) ... ok
test_list (main.ContainerTestCase) ... ok
test_sets (main.ContainerTestCase) ... ok
test_tuple (main.ContainerTestCase) ... ok
test_exceptions (main.ExceptionTestCase) ... ok
test_bug_5888452 (main.BugsTestCase) ... ok
test_exact_type_match (main.BugsTestCase) ... ok
test_fuzz (main.BugsTestCase) ... ok
test_loads_recursion (main.BugsTestCase) ...
^^^

python_d.exe crashes at this point with a stack overflow. Just
capturing the issue for now. Will investigate shortly and hopefully
provide a patch.

Traced the problem down to the following minimal code snippet:

import marshal
s = 'c' + ('X' * 4*4) + '{' * 2**20
marshal.loads(s)

When Python/marshal.c:18 MAX_MARSHAL_STACK_DEPTH is 2000 (which is what
it is currently), marshal.loads() eventually overflows the stack in
r_object(). There is a check in r_object() to avoid this though:

  if (p->depth > MAX_MARSHAL_STACK_DEPTH) {
      p->depth--;
      PyErr_SetString(PyExc_ValueError, "recursion limit exceeded");
      return NULL;
  }

On Windows x64, a value of 1964 raises the recursion limit exception
above (which is what test_marshal is expecting). With a value of 1965,
a C stack overflow exception is raised.

So, MAX_MARSHAL_STACK_DEPTH needs to be <= 1964 in order to prevent this
particular code from overflowing the stack on Win64 before we can raise
a Python recursion limit exception.

Was there any science behind choosing 2000 as the current value? Should
a new value (e.g. 1500) be provided for only Win64, leaving everyone
else at 2000? Interesting that Linux/BSD etc on AMD64 don't run into this.

The situation was the same on win32 with the VC2005 debug build, some
months ago.
I think this is because of the extra stack checks added with "recent"
versions of Visual Studio, together with the fact that local variables
are not shared when optimizations are disabled.

Can you try to raise the stack size on x64 builds? If 2Mb is enough for
32bit, 4Mb should be good in your case.

Fixed in r61509; bumped the stack size from 200000 to 210000, which
stops the stack overflow from being triggered.

I had the exact same problem when compiling the Python 2.7 branch with Visual Studio 2010.

In debug mode and 64 bits, python will crash on test_loads_recursion in test_marshal and on the code provided in msg63536.

Could you please reopen this issue?

I don't have the issue anymore when bumping the stack size to 2300000.

The 'official' PSF build for 2.7 (and 3.x) is done with VS2008, so this may not be the last problem you have. I am pretty sure this will never change for 2.7, but that is not to say we cannot make a simple change to accommodate VS2010. I am sure we will eventually update VS for 3.x. Can you test 3.2 or even better 3.3?

Thanks Terry, I am aware of that.
We are working on making Python work with VS2010 in bpo-13210.

I will check with the py3k branch soon and report here if the same problem applies.

Closing issue; this has been fixed.

Reopening because I think this is again a problem for Win64 and 3.x. The Win64 buildbots always seem to crash on test_marshal (and I do too).

It appears to be BugsTestCase.test_loads_2x_code() which crashes, which is virtually the same as test_loads_recursion().

Closing because this is caused by bpo-17206 and is already discussed there.