classification
Title: MMAP: Bus error (core dump) under heavy read/write
Type: crash Stage: resolved
Components: IO Versions: Python 2.7
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jcea, neologix, pitrou, rbhkamal, sbt, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2013-10-21 18:24 by rbhkamal, last changed 2013-10-22 02:43 by jcea. This issue is now closed.

Messages (5)
msg200820 - (view) Author: Ribhi Kamal (rbhkamal) Date: 2013-10-21 18:24
I was about to use memory mapping for something when I discovered that the code example posted on python.org causes python to core dump. %100 reproducible

The steps are really simple. 
1- Create a file and map it using mmap.mmap
2- In a while loop, contintously read the file
3- From another process, like bash, continuously write to the file.

After about 5 iterations from the reader, python core dumps.

Reader code (python):
#!/usr/bin/python2.7

import mmap
import time

with open("hello.txt", "wb") as f:
    f.write("Hello Python! 1234123412341234\n")

with open("hello.txt", "r+b") as f:
    # memory-map the file, size 0 means whole file
    mm = mmap.mmap(f.fileno(), 0)
    count=0
    while count < 100:
       mm.seek(0)
       print mm.readline()
       time.sleep(0.1)
       count = count + 1

    # close the map
    mm.close()


Writer code (linux shell/bash):
#!/bin/bash
count=0
while true
do
 ((count++))
 echo $count > hello.txt
done


Now run the reader, then launch the writer in a terminal. In my case I get the following output:
>110
>
>462
>
>Bus error (core dumped)


Python 2.7.3
Linux 3.2.0-54-generic #82-Ubuntu SMP Tue Sep 10 20:09:12 UTC 2013 i686 i686 i386 GNU/Linux
Ubuntu 12.04.3 LTS
Intel(R) Core(TM)2 CPU          6300  @ 1.86GHz
msg200821 - (view) Author: Ribhi Kamal (rbhkamal) Date: 2013-10-21 18:30
Code dump attached
msg200822 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-10-21 18:32
Confirm on 2.7 and 3.x.
msg200823 - (view) Author: Charles-Fran├žois Natali (neologix) * (Python committer) Date: 2013-10-21 18:33
> 3- From another process, like bash, continuously write to the file.

That's the problem: reducing (actually truncating in your case) a file currently mmaped *can only result in a core dump*: when you try to read from a location in memory which doesn't correspond anymore to a position in the file (since it got truncated), which value could the kernel return?

The page you're trying to read from isn't mapped anymore, so you get a segfault.

Apparently it's even documented in the main page:
"""
The effect of changing the size of the underlying file of a mapping on the pages that correspond to added or removed regions of
       the file is unspecified.
"""
msg200825 - (view) Author: Ribhi Kamal (rbhkamal) Date: 2013-10-21 18:36
I figured I was doing something wrong... sorry about that
History
Date User Action Args
2013-10-22 02:43:51jceasetnosy: + jcea
2013-10-21 18:37:14rbhkamalsetfiles: - core.gz
2013-10-21 18:36:38rbhkamalsetstatus: open -> closed
2013-10-21 18:36:19rbhkamalsetstatus: closed -> open
versions: - Python 3.3, Python 3.4
messages: + msg200825

components: - Library (Lib)
resolution: not a bug ->
2013-10-21 18:33:57neologixsetstatus: open -> closed
resolution: not a bug
messages: + msg200823

stage: resolved
2013-10-21 18:32:38serhiy.storchakasetversions: + Python 3.3, Python 3.4
nosy: + serhiy.storchaka

messages: + msg200822

components: + Library (Lib)
2013-10-21 18:30:38rbhkamalsetfiles: + core.gz

messages: + msg200821
2013-10-21 18:26:15vstinnersetnosy: + pitrou, vstinner, neologix, sbt
2013-10-21 18:24:45rbhkamalcreate