This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Assertion with very long lists
Type: Stage:
Components: Interpreter Core Versions:
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: gvanrossum, jalet, nnorwitz, tim.peters
Priority: normal Keywords:

Created on 2002-05-29 10:12 by jalet, last changed 2022-04-10 16:05 by admin. This issue is now closed.

Messages (14)
msg10952 - (view) Author: Jerome Alet (jalet) Date: 2002-05-29 10:12
I've tried to import a module which only define a list
of 170000 elements (a word list) which defines an
element per line, so the module also has 170000 lines
in it. Here's the error :

--- CUT ---
Python 2.1.3 (#1, Apr 20 2002, 10:14:34) 
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "copyright", "credits" or "license" for more
>>> import dictionnaire
python: ../Python/ceval.c:695: eval_code2: Assertion
`(stack_pointer - f->f_valuestack) <= f->f_stacksize'
--- CUT ---

this seems to be the same problem as bug id 210647 but
there's no segfault.

the module is attached to this bug report.
msg10953 - (view) Author: Jerome Alet (jalet) Date: 2002-05-29 10:14
Logged In: YES 

module too long to be attached. ask it by email at if needed.
msg10954 - (view) Author: Jerome Alet (jalet) Date: 2002-05-29 12:42
Logged In: YES 

I've done more tests, the bug only occurs if every line has
a backslash at its end :

liste = [ \
            "item0", \
            "item1", \
            # ... many many lines
            "itemN" ]

and it works if I remove the \ from each end of line.

msg10955 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-05-31 22:32
Logged In: YES 

I think there are several problems here.
I was able to narrow down at least one problem to a long list.
It doesn't seem to have anything to do w/backslashes.

There seems to be an interaction with importing the module.
If the long list is in __main__, the problem does not occur.
The problem only occurs when the list is in an imported module
and only after importing it the second time (after the .pyc
is created).

If a list has 32768+ items there seems to be a memory problem.
There is an exception, then a crash.  Here's the exception:

TypeError: unsupported operand type(s) for &: 'str' and 'str'

Here's a stack trace against CVS version:

#0  chunk_free (ar_ptr=0x401b1620, p=0x8156a10) at malloc.c:3180
#1  0x400fdbf4 in __libc_free (mem=0x8156a18) at malloc.c:3154
#2  0x0805c0db in list_dealloc (op=0x401c4eac) at
#3  0x080c7618 in PyDict_SetItem (op=0x401bd79c,
    value=0x80e185c) at Objects/dictobject.c:373
#4  0x080c93f9 in PyDict_SetItemString (v=0x401bd79c,
key=0x80d9070 "path", 
    item=0x80e185c) at Objects/dictobject.c:1895
#5  0x08099ad8 in PyImport_Cleanup () at Python/import.c:286
#6  0x080a266d in Py_Finalize () at Python/pythonrun.c:228
#7  0x08053766 in Py_Main (argc=2, argv=0xbffff904) at
msg10956 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-06-01 00:06
Logged In: YES 

I found at least part of the problem, perhaps everything.
The co_stacksize > 32767 and only a short value is marshalled
to the .pyc.

See Python/marshal.c line 246.

Not sure how to fix yet.
msg10957 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-06-01 00:17
Logged In: YES 

Got a fix.  Patch attached.  
It will probably have to be changed,
since I don't remember what else needs to be
done when the MAGIC is changed.
msg10958 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-06-01 02:04
Logged In: YES 

Final patch attached.
msg10959 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-06-01 14:56
Logged In: YES 

Added a patch for 2.2 (2.1) is basically the same.
No test for 2.2/2.1 though.
Could someone please review before checkin?
msg10960 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2002-06-01 18:14
Logged In: YES 

Assuming the patch you're talking about is the one named 
long-list-2.2.diff, looks good to me, so Accepted and 
assigned back to you.  Thanks!
msg10961 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-06-01 18:28
Logged In: YES 

Thanks Tim.
Checked in for 2.1/2.2 as and 

Could you or someone also review the other patch for current
That patch has a test and is different ([rw]_short ->
msg10962 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-06-04 17:20
Logged In: YES 

Tim, could you also review long-list.diff.  This is the
patch against CVS version.
It changes what is marshalled to disk.  Thanks.
msg10963 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-06-13 22:08
Logged In: YES 

Guido, could you review this patch before checking in?
msg10964 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2002-06-14 00:14
Logged In: YES 

This looks good to me. You might want to increase the size
of the list in the testcase so that it actually crashes for
non-debug Python without the fix; I could only get it to
fail an assertion in debug mode.
msg10965 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-06-14 01:08
Logged In: YES 

I increased the list size in the test to 65000 which crashed
on my box.

Checked in as:
  Python/import.c  2.206
  Python/marshal.c  1.72  Lib/test/  1.12
Date User Action Args
2022-04-10 16:05:21adminsetgithub: 36661
2002-05-29 10:12:56jaletcreate