This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multiprocessing.shared_memory: MacOS crashes by running attached Python code
Type: crash Stage: patch review
Components: C API, Documentation, macOS Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: christian.heimes, davin, docs@python, ned.deily, pitrou, ronaldoussoren, vinay0410
Priority: normal Keywords: patch

Created on 2020-02-08 14:18 by vinay0410, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 21877 open vinay0410, 2020-08-14 09:11
Messages (9)
msg361629 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-02-08 14:18
Consider the following python Code.

```
from multiprocessing.shared_memory import SharedMemory
shm = SharedMemory(name='test-crash', create=True, size=1000000000000000000)
```

This causes macOS Catalina, Mojave to freeze and then crash. Although, this works fine on ubuntu.

After, debugging I realised that this is due to the ftruncate call. I could replicate the same by calling os.ftruncate and also using ftruncate in C code.

Following C++ code also crashes, which confirms that ftruncate in macOS is broken:

```
#include <fcntl.h>
#include <sys/shm.h>
#include <sys/stat.h>
#include <iostream>
#include <sys/mman.h>
#include <unistd.h>
#include <sys/types.h>
#include <errno.h>
#include <time.h>

int main() {

    int shm_fd = shm_open("/test-shm2", O_CREAT | O_RDWR, 0666);

    if (shm_fd == -1) {
        throw "Shared Memory Object couldn't be created or opened";
    }

    int rv = ftruncate(shm_fd, (long long)1000000000000000000);

}
```

Should python, in any way handle this, so as to prevent any crashes using python code.
msg361924 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2020-02-12 23:06
My sense is that it would be nice if we can catch this before ftruncate does something nasty.

Where else is ftruncate used in CPython that this could similarly trigger a problem?  How is it handled there (or not)?
msg373806 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-07-17 09:41
Hi, I tried replicating this by truncating normal files but that doesn't crash. The above mentioned call of ftruncate only crashes for when the file descriptor passed points to a shared memory segment.
And only, multiprocessing.shared_memory is currently creating shared_memory using _posixshmem.shm_open.

So, it can be fixed in ftruncate implementation or this can also be handled by multiprocessing.shared_memory.

Please let me know your thoughts about the same.
msg375229 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-08-12 12:49
I expect that this is more a problem with how memory management works on macOS, the freeze and crash is likely an indication of eager allocation of memory by the system.

It is far from sure if implementing a workaround is feasible, the upper limit for safe calls to ftruncate likely depends on the amount of RAM in the system.

BTW. The sample code tries to allocate well over a petabyte of memory, how realistic is that code?  It might be good enough to document that using very large amounts of memory with multiprocessing.shared_memory causes problems on macOS.
msg375245 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-08-12 13:50
I have 8GB of ram and 128 GB of hard disk.

Now, creating a shared memory segment of size 10^12 (1 terabyte) somehow succeeds.

Creating a shared memory segment of 10^15 (1 petabyte), mmap (not ftruncate) throws an error stating cannot allocate memory.

Creating a shared memory segment of 10^18 (1 exabyte), causes the system to crash.

Now, I understand that this should be documented for a genuine user.

But, if documented this can be used by malicious softwares using python to crash systems abruptly.

Also, I understand that this is an issue with macos, but shouldn't python handle this so that atleast python's APIs are safe.

Creating shared memory segments of size 1 exabyte are not reasonable, but if some makes a mistake then, we must throw an error instead of a crash.

Also, can we set a max limit on creating shared memory segments to 1TB ?
because no one would genuinily need to create a segment of that size on a single machine.
msg375249 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-08-12 14:07
A workaround in the implementation of multiprocessing.SharedMemory is IMHO acceptable, tweaking os.ftruncate less so.

Note that a Python script can already cause problems on systems by using APIs as intended (such as using shutil.rmtree on the user's home directory).

There's balance between compensating for platform deviancies and having clean and performant implementation. 

BTW. We should file an issue with Apple about this. I'll do so when I've had time to crash a test VM using the C code you provided.
msg381221 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-11-17 09:37
I'm strong -1 on any kind of arbitrary memory limit. Python is used on all sorts of hardware from RPi to super computers. 1 TB sounds like a lot, but it really is not. These days you can buy workstation computers with more than 1 TB RAM.

I would be ok with a limit of 2**48 on X86_64 platforms. Current X86_64 CPUs have a virtual address size limit of 48 bits. Other hardware platforms may have a similar limit. I don't have access to AArch64 or PPC64 to verify the address size.
msg381222 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-11-17 10:24
Having thought about this a little more: I agree, we shouldn’t hard code a limit.  I’m not against working around misfeatures, but in this case that’s hard to do: Even if we’d limit the size of a single shared memory segment the user can create a number of them to once again use enough memory to crash the machine.

IMHO We should document that this can happen on macOS (preferable with some indication of how much shared memory can safely be used). I’ve therefore added the “Documentation” component.
msg381223 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-11-17 10:55
I’ve filed an issue with Apple about this: FB8903019
History
Date User Action Args
2022-04-11 14:59:26adminsetgithub: 83765
2020-11-17 10:55:29ronaldoussorensetmessages: + msg381223
2020-11-17 10:24:46ronaldoussorensetnosy: + docs@python
messages: + msg381222

assignee: docs@python
components: + Documentation
2020-11-17 09:37:41christian.heimessetmessages: + msg381221
2020-08-14 09:11:34vinay0410setkeywords: + patch
stage: patch review
pull_requests: + pull_request21002
2020-08-12 14:07:46ronaldoussorensetmessages: + msg375249
2020-08-12 13:50:14vinay0410setmessages: + msg375245
2020-08-12 12:49:20ronaldoussorensetmessages: + msg375229
2020-08-12 12:19:24ned.deilysetnosy: + ned.deily, ronaldoussoren
components: + macOS
2020-07-20 09:16:20vinay0410setnosy: + pitrou
2020-07-17 09:41:53vinay0410setnosy: + christian.heimes
messages: + msg373806
2020-02-12 23:06:19davinsetmessages: + msg361924
2020-02-10 10:58:45vstinnersettitle: MacOS crashes by running attached Python code -> multiprocessing.shared_memory: MacOS crashes by running attached Python code
2020-02-10 10:58:26vstinnersetnosy: - vstinner
2020-02-08 14:51:44SilentGhostsetnosy: + vstinner, davin
2020-02-08 14:18:41vinay0410settype: crash
2020-02-08 14:18:24vinay0410create