This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vinay0410
Recipients davin, eryksun, pitrou, vinay0410
Date 2019-09-09.12:46:45
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
Hi @davin,
I researched on lots of approaches to solve this problem, and I have listed down some of the best ones.

1. As Eryk Sun suggested initially to use advisory locking to implement a reference count type of mechanism. I implemented this in the current Pull Request already open. And it works great on most platforms, but fails on MacOS. This is because MacOS makes no file-system entry like Linux in /dev/shm. In fact it makes no filesystem entry for the created shared memory segment. Therefore this won't work.

2. I thought of creating a manual file entry for the created shared memory segment in /tmp in MacOS. This will work just fine unless the user manually changes the permissions of /tmp directory on MacOS. And we will have to rely on the fact that /tmp is writable if we use this approach.

3. Shared Semaphores: This is a very interesting approach to implement reference count, where a semaphore is created corresponding to every shared memory segment, which keeps reference count of the shared memory.
Resource Tracker will clear the shared memory segment and the shared semaphore as soon as the value of the shared semaphore becomes 0. 

The only problem with this approach is to decide the name of shared semaphore. We will have to define a standardised way to get extract shared semaphore's name from shared segment's name.

For instance, shared_semaphore_name = 'psem' + shared_segment_name.
This can cause problems if a semaphore with shared_semaphore_name is already present.

This could be solved by taking any available name and storing it inside shared memory segment upon creation, and then extracting this name to open the shared semaphore.

4. Another way could be to initialize a semaphore by sem_init() inside the shared memory segment. For example the first 4 bytes can be reserved for semaphore. But, since MacOS has deprecated sem_init(), it wouldn't be a good practice.

5. Atomic Calls: I think this is the most simple and best way to solve the above problem. We can reserve first 4 bytes for an integer which is nothing but the reference count of shared memory, and to prevent data races we could use atomic calls to update it.
gcc has inbuilt support for some atomic operations. I have tested them using processes by updating them inside shared memory. And they work great.

Following are some atomic calls:

type __sync_add_and_fetch (type *ptr, type value, ...)
type __sync_sub_and_fetch (type *ptr, type value, ...)
type __sync_or_and_fetch (type *ptr, type value, ...)

Documentation of these can be found at below links:
Date User Action Args
2019-09-09 12:46:46vinay0410setrecipients: + vinay0410, pitrou, eryksun, davin
2019-09-09 12:46:46vinay0410setmessageid: <>
2019-09-09 12:46:46vinay0410linkissue37754 messages
2019-09-09 12:46:45vinay0410create