classification
Title: shared_semaphores cannot be shared across unrelated processes
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Ido Michael, davin, pitrou, taleinat, vinay0410
Priority: normal Keywords:

Created on 2019-09-05 05:01 by vinay0410, last changed 2020-06-19 09:55 by vinay0410.

Messages (11)
msg351176 - (view) Author: Vinay Sharma (vinay0410) * Date: 2019-09-05 05:01
Currently, shared semaphores can only be created, and existing semaphores can't be opened. Shared semaphores are opened using the following command.
```
sem_open(name, O_CREAT | O_EXCL, 0600, val)
```
This will raise error if a semaphore which already exists.
This behaviour works well when the file descriptors of these semaphores can be shared with children processes.

But, it doesn't work when an unrelated process which needs access to shared semaphore tries to open it.
msg351214 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-09-05 17:26
Please give an example of how this is a problem in using Python. You don't give any context here.
msg351225 - (view) Author: Vinay Sharma (vinay0410) * Date: 2019-09-06 03:35
Let's say I have two processes which are accessing a shared resource (let's say shared memory). Now, these two processes will need some kind of synchronisation mechanism, to be able to access the resource properly, and avoid race condition.

Currently, shared semaphores can be shared with child processes, but there is no way of accessing a named shared semaphore by two separate processes.

Although, the semaphore created by default is a named semaphore, and has the potential to be accessed by multiple processes, but because of the flags, it can't be opened by any other process.

This is a feature request, and not a bug, therefore I have selected type as enhancement.
msg352361 - (view) Author: Vinay Sharma (vinay0410) * Date: 2019-09-13 16:41
A common use for the same can be shared memory. Currently shared memory can be used by unrelated processes, but there is no mechanism as such to synchronise them at the moment.
msg353455 - (view) Author: Ido Michael (Ido Michael) * Date: 2019-09-28 14:32
@ned.deily any thoughts?

I can start working on it.

Ido
msg362486 - (view) Author: Ido Michael (Ido Michael) * Date: 2020-02-23 01:54
@taleinat what do you think?
msg362493 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2020-02-23 07:44
I'm not convinced there is enough real need for this to justify implementing and supporting it.

Using shared memory wouldn't be possible with built-in Python objects; one would have to use a special-purpose library for this, or perhaps use something like ctypes. It seems to me that in any case, once someone is doing those, they might as well use those same tools to also create and manage shared resources such as shared memory.

Therefore, even if we allow creating such shared resources, it doesn't seem to me that they will be very usable without a lot of extra supporting features.

In other words, I'd like to see at least one convincing use-case for this before considering any implementation.
msg362495 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-02-23 08:58
Hi,
I think a use case for this is https://docs.python.org/3/library/multiprocessing.shared_memory.html

If not, can you please suggest a way to synchronise the above across unrelated processes.
msg370322 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2020-05-29 16:11
Sorry this hasn't been followed up. I think this would best be discussed a bit on the Python-Ideas mailing list. Please send an email there suggesting this feature, and link to the discussion thread in the mailing list archives here.
msg370764 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-06-05 15:21
As suggested I have sent a mail to Python Ideas regarding this issue.
Link to Python Ideas Archive: https://mail.python.org/archives/list/python-ideas@python.org/thread/X4AKFFMYEKW6GFOUMXMOJ2OBINNY2Q6L/
msg371863 - (view) Author: Vinay Sharma (vinay0410) * Date: 2020-06-19 09:55
Hi @taleinat, I had sent a mail to the python mailing list as suggested. And till now it has received rather positive comments, in the sense that people who responded see this is a useful feature to be integrated into python.
History
Date User Action Args
2020-06-19 09:55:04vinay0410setmessages: + msg371863
2020-06-05 15:21:51vinay0410setmessages: + msg370764
2020-05-29 16:11:12taleinatsetmessages: + msg370322
2020-02-23 08:58:18vinay0410setmessages: + msg362495
2020-02-23 07:44:13taleinatsetmessages: + msg362493
2020-02-23 01:54:56Ido Michaelsetnosy: + taleinat
messages: + msg362486
2019-09-28 16:12:39ned.deilysetnosy: + pitrou, davin
2019-09-28 14:32:08Ido Michaelsetnosy: + Ido Michael
messages: + msg353455
2019-09-13 23:09:17ned.deilysetnosy: - ned.deily
2019-09-13 16:41:04vinay0410setmessages: + msg352361
2019-09-06 03:35:22vinay0410setmessages: + msg351225
2019-09-05 17:26:24ned.deilysetnosy: + ned.deily
messages: + msg351214
2019-09-05 05:01:29vinay0410settype: enhancement
2019-09-05 05:01:08vinay0410create