This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: mmap.resize and offset
Type: crash Stage:
Components: Extension Modules Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: ocean-city, tim.golden
Priority: normal Keywords: patch

Created on 2009-02-16 17:21 by ocean-city, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
fix_offset_and_resize.patch ocean-city, 2009-02-17 12:00
Messages (6)
msg82253 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-02-16 17:21
This is first time to look at mmap module, so sorry if I'm saying
totally wrong thing. (I noticed this when I saw issue2733) I think the
behavior of mmap_resize_method is unclear when mapping object is created
with offset > 0.

From view of other functions, it seems that self->offset is offset from
beginning of the file, and self->size is the size of mapped area from
self->offset. If so,

#if SIZEOF_SIZE_T > 4
    newSizeHigh = (DWORD)((self->offset + new_size) >> 32);
    newSizeLow = (DWORD)((self->offset + new_size) & 0xFFFFFFFF);
#else
    newSizeHigh = 0;
    newSizeLow = (DWORD)new_size; /* shouldn't add self->offset? this is
not doing same thing as SIZEOF_SIZE_T > 4 */
#endif

And on unix part,
    newmap = mremap(self->data, self->size, new_size, MREMAP_MAYMOVE);
self->offset is totally ignored. I think when self->offset > 0,
something wrong happens.

And comment above function definition saids,

 / Is this really necessary?  This could easily be done
 / from python by just closing and re-opening with the
 / new size?

I think this function is not tested well. There is no test about
resize+offset in test_mmap.py.
msg82254 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2009-02-16 17:25
Have a look at issue 2733

  http://bugs.python.org/issue2733

where I've just proposed a patch in this area. I'm also
not sure exactly what's going on, but I have patched what
I believe is a linked pair of bugs in that code.
msg82256 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-02-16 17:37
Yes, I noticed this issue while investigating issue2733. But sorry, I
couldn't reproduce the bug on my machine. Even so, LPDWORD arg for
SetPointer can be modified, so your patch looks correct. (There is
SetPointerEx which can make the code simpler, but this function seems
not available on WinCE)
msg82300 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-02-17 05:35
I investigated more. Following code crashed with bus error on coLinux.

import mmap

def main():
    align = mmap.ALLOCATIONGRANULARITY
    with open("a.txt", "w") as f:
        f.write("0" * align)
        f.write("1" * align)
        f.write("2" * align)
    with open("a.txt", "r+") as f:
        m = mmap.mmap(f.fileno(), align, offset=align*2)
    m.resize(1)
    print m[0] # bus error

if __name__ == '__main__':
    main()

On windows, failed with AccessDenied. (32bit Win2000)
msg82323 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-02-17 12:00
Here is a patch.
msg82337 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-02-17 13:19
I hope this is obvious too. Fixed in r69718.
History
Date User Action Args
2022-04-11 14:56:45adminsetgithub: 49532
2009-02-17 13:19:16ocean-citysetstatus: open -> closed
resolution: fixed
messages: + msg82337
2009-02-17 12:00:04ocean-citysetfiles: + fix_offset_and_resize.patch
keywords: + patch
messages: + msg82323
2009-02-17 05:35:49ocean-citysettype: behavior -> crash
messages: + msg82300
2009-02-16 17:37:40ocean-citysetmessages: + msg82256
2009-02-16 17:25:40tim.goldensetnosy: + tim.golden
messages: + msg82254
2009-02-16 17:21:47ocean-citycreate