classification
Title: multiprocessing.shared_memory: MacOS crashes by running attached Python code
Type: crash Stage: patch review
Components: C API, macOS Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, davin, ned.deily, pitrou, ronaldoussoren, vinay0410
Priority: normal Keywords: patch

Created on 2020-02-08 14:18 by vinay0410, last changed 2020-08-14 09:11 by vinay0410.

Pull Requests
URL Status Linked Edit
PR 21877 open vinay0410, 2020-08-14 09:11
Messages (6)
msg361629 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-02-08 14:18
Consider the following python Code.

```
from multiprocessing.shared_memory import SharedMemory
shm = SharedMemory(name='test-crash', create=True, size=1000000000000000000)
```

This causes macOS Catalina, Mojave to freeze and then crash. Although, this works fine on ubuntu.

After, debugging I realised that this is due to the ftruncate call. I could replicate the same by calling os.ftruncate and also using ftruncate in C code.

Following C++ code also crashes, which confirms that ftruncate in macOS is broken:

```
#include <fcntl.h>
#include <sys/shm.h>
#include <sys/stat.h>
#include <iostream>
#include <sys/mman.h>
#include <unistd.h>
#include <sys/types.h>
#include <errno.h>
#include <time.h>

int main() {

    int shm_fd = shm_open("/test-shm2", O_CREAT | O_RDWR, 0666);

    if (shm_fd == -1) {
        throw "Shared Memory Object couldn't be created or opened";
    }

    int rv = ftruncate(shm_fd, (long long)1000000000000000000);

}
```

Should python, in any way handle this, so as to prevent any crashes using python code.
msg361924 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2020-02-12 23:06
My sense is that it would be nice if we can catch this before ftruncate does something nasty.

Where else is ftruncate used in CPython that this could similarly trigger a problem?  How is it handled there (or not)?
msg373806 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-07-17 09:41
Hi, I tried replicating this by truncating normal files but that doesn't crash. The above mentioned call of ftruncate only crashes for when the file descriptor passed points to a shared memory segment.
And only, multiprocessing.shared_memory is currently creating shared_memory using _posixshmem.shm_open.

So, it can be fixed in ftruncate implementation or this can also be handled by multiprocessing.shared_memory.

Please let me know your thoughts about the same.
msg375229 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-08-12 12:49
I expect that this is more a problem with how memory management works on macOS, the freeze and crash is likely an indication of eager allocation of memory by the system.

It is far from sure if implementing a workaround is feasible, the upper limit for safe calls to ftruncate likely depends on the amount of RAM in the system.

BTW. The sample code tries to allocate well over a petabyte of memory, how realistic is that code?  It might be good enough to document that using very large amounts of memory with multiprocessing.shared_memory causes problems on macOS.
msg375245 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-08-12 13:50
I have 8GB of ram and 128 GB of hard disk.

Now, creating a shared memory segment of size 10^12 (1 terabyte) somehow succeeds.

Creating a shared memory segment of 10^15 (1 petabyte), mmap (not ftruncate) throws an error stating cannot allocate memory.

Creating a shared memory segment of 10^18 (1 exabyte), causes the system to crash.

Now, I understand that this should be documented for a genuine user.

But, if documented this can be used by malicious softwares using python to crash systems abruptly.

Also, I understand that this is an issue with macos, but shouldn't python handle this so that atleast python's APIs are safe.

Creating shared memory segments of size 1 exabyte are not reasonable, but if some makes a mistake then, we must throw an error instead of a crash.

Also, can we set a max limit on creating shared memory segments to 1TB ?
because no one would genuinily need to create a segment of that size on a single machine.
msg375249 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-08-12 14:07
A workaround in the implementation of multiprocessing.SharedMemory is IMHO acceptable, tweaking os.ftruncate less so.

Note that a Python script can already cause problems on systems by using APIs as intended (such as using shutil.rmtree on the user's home directory).

There's balance between compensating for platform deviancies and having clean and performant implementation. 

BTW. We should file an issue with Apple about this. I'll do so when I've had time to crash a test VM using the C code you provided.
History
Date User Action Args
2020-08-14 09:11:34vinay0410setkeywords: + patch
stage: patch review
pull_requests: + pull_request21002
2020-08-12 14:07:46ronaldoussorensetmessages: + msg375249
2020-08-12 13:50:14vinay0410setmessages: + msg375245
2020-08-12 12:49:20ronaldoussorensetmessages: + msg375229
2020-08-12 12:19:24ned.deilysetnosy: + ned.deily, ronaldoussoren
components: + macOS
2020-07-20 09:16:20vinay0410setnosy: + pitrou
2020-07-17 09:41:53vinay0410setnosy: + christian.heimes
messages: + msg373806
2020-02-12 23:06:19davinsetmessages: + msg361924
2020-02-10 10:58:45vstinnersettitle: MacOS crashes by running attached Python code -> multiprocessing.shared_memory: MacOS crashes by running attached Python code
2020-02-10 10:58:26vstinnersetnosy: - vstinner
2020-02-08 14:51:44SilentGhostsetnosy: + vstinner, davin
2020-02-08 14:18:41vinay0410settype: crash
2020-02-08 14:18:24vinay0410create