This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Valgrind error when running Python command within Vim
Type: Stage:
Components: Library (Lib) Versions: Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: dominiko, mark.dickinson
Priority: normal Keywords:

Created on 2010-02-21 14:04 by dominiko, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
valgrind.log.gz dominiko, 2010-02-21 14:04 valgrind log file showing access to freed memory
Messages (5)
msg99660 - (view) Author: Dominique Pellé (dominiko) Date: 2010-02-21 14:04
I built Vim-7.2.368 editor with python interpreter using
Python-2.6.4 library on Linux x86.

When I run a python command (any command, it does not matter)
in the Vim editor with Valgrind memory checker, I see valgrind
errors within the Python library.  It looks to me like they
are errors within the Python library and not errors in Vim.

I attach the valgrind error log file.

Steps to reproduce the bug:

1/ Download & compile Vim with python interpreter

  $ hg clone https://vim.googlecode.com/hg/ vim
  $ cd vim
  $ ./configure --enable-pythoninterp --with-python-config-dir=/usr/local/lib/python2.6/config
  $ make

(you may also want to tweak vim/src/Makefile to compile with -O0 -g
to have more interesting stacks in valgrind logs)

2/ Run a Python command within Vim with Valgrind memory checker: 

  $ cd vim/src
  $ valgrind --log-file=valgrind.log ./vim -c ':python "foo=0"'

3/ Observe in valgrind.log the errors in Python lib  

The stack is quite deep in the errors and Valgrind memory checker has a limit to dump at most 50 functions in stack frame. I increased that limit to 100 to be able to see the full stack trace as in attached log file "valgrind.log".

In the fist error in attached valgrind.log, memory is used in a
buffer that has already been freed. Looking at the Python
code, this function does not make much sense to me:

1040 PyObject *
1041 PyMarshal_ReadLastObjectFromFile(FILE *fp)
1042 {
1043 /* 75% of 2.1's .pyc files can exploit SMALL_FILE_LIMIT.
1044  * REASONABLE_FILE_LIMIT is by defn something big enough for Tkinter.pyc.
1045  */
1046 #define SMALL_FILE_LIMIT (1L << 14)
1047 #define REASONABLE_FILE_LIMIT (1L << 18)
1048 #ifdef HAVE_FSTAT
1049         off_t filesize;
1050 #endif
1051 #ifdef HAVE_FSTAT
1052         filesize = getfilesize(fp);
1053         if (filesize > 0) {
1054                 char buf[SMALL_FILE_LIMIT];
1055                 char* pBuf = NULL;
1056                 if (filesize <= SMALL_FILE_LIMIT)
1057                         pBuf = buf;
1058                 else if (filesize <= REASONABLE_FILE_LIMIT)
1059                         pBuf = (char *)PyMem_MALLOC(filesize);
1060                 if (pBuf != NULL) {
1061                         PyObject* v;
1062                         size_t n;
1063                         /* filesize must fit into an int, because it
1064                            is smaller than REASONABLE_FILE_LIMIT */
1065                         n = fread(pBuf, 1, (int)filesize, fp);
1066                         v = PyMarshal_ReadObjectFromString(pBuf, n);
1067                         if (pBuf != buf)
1068                                 PyMem_FREE(pBuf);
1069                         return v;
1070                 }
1071 
1072         }
1073 #endif
1074         /* We don't have fstat, or we do but the file is larger than
1075          * REASONABLE_FILE_LIMIT or malloc failed -- read a byte at a time.
1076          */
1077         return PyMarshal_ReadObjectFromFile(fp);
1078 
1079 #undef SMALL_FILE_LIMIT
1080 #undef REASONABLE_FILE_LIMIT
1081 }

Memory is allocated for pBuf at line marshal.c:1059.
Then at line marshall.c:1066 v= PyMarshal_ReadObjectFromString(pBuf, n);
is called. 

The v structure contains pointer to the pBuf buffer (see line marshall.c:1103). Then pBuf is freed at marshall.c:1068 and v is returned.

So v which is returned contains a pointer "v.ptr" to buffer
that has just been freed. That looks wrong to me. What's the
point of v containing an address to something freed?

Looking at the latest version of Python/marshal.c in SVN,
this function has not been changed since 2.6.4:

http://svn.python.org/projects/python/trunk/Python/marshal.c
msg99662 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-02-21 14:14
The code you identify looks okay to me:  in PyMarshal_ReadObjectFromString, isn't it only the temporary variable rf that has a pointer to the string?

Have you read Misc/README.valgrind in the Python source?
msg99663 - (view) Author: Dominique Pellé (dominiko) Date: 2010-02-21 14:36
> Have you read Misc/README.valgrind in the Python source?

No, I had not see this file.
Thanks for pointing it to me.

I've just read it, reconfigured & recompiled Python-2.6.4 with:

./configure --without-pymalloc

It now runs without Valgrind error.

Sorry for the noise. 
This issue can thus be closed
msg99664 - (view) Author: Dominique Pellé (dominiko) Date: 2010-02-21 14:38
Closed: this was not a bug, I had to build Python lib with "configure --without-pymalloc" to avoid valgrind errors.
msg99666 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-02-21 15:08
Thanks for the update!
History
Date User Action Args
2022-04-11 14:56:57adminsetgithub: 52222
2010-02-21 15:08:31mark.dickinsonsetresolution: not a bug
2010-02-21 15:08:00mark.dickinsonsetmessages: + msg99666
2010-02-21 14:38:15dominikosetstatus: open -> closed

messages: + msg99664
2010-02-21 14:36:51dominikosetmessages: + msg99663
2010-02-21 14:14:11mark.dickinsonsetnosy: + mark.dickinson
messages: + msg99662
2010-02-21 14:04:20dominikocreate