classification
Title: semaphore errors on AIX 7.1
Type: behavior Stage: resolved
Components: Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: haypo, neologix, pitrou, python-dev, reshmi.george, sable
Priority: normal Keywords: needs review, patch

Created on 2011-06-20 08:56 by reshmi.george, last changed 2011-07-21 18:01 by neologix. This issue is now closed.

Files
File name Uploaded Description Edit
aix_broken_sem.diff neologix, 2011-06-23 16:58 review
Messages (11)
msg138693 - (view) Author: reshmi george (reshmi.george) Date: 2011-06-20 08:56
The same problem that was reported in issue 9700 is appearing on AIX 7.1. 
The following message has been seen when running multi-process python program:
sem_trywait: Permission denied
sem_post: Permission denied
sem_wait: Permission denied
sem_post: Permission denied


It can be easily corrected by defining HAVE_BROKEN_POSIX_SEMAPHORES for AIX 7, like it is done for AIX 6.
msg138871 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-06-23 16:58
Patch attached.
I've used a wildcard so that it won't break when AIX 8 goes out (and it's POSIX).
msg138873 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-23 21:25
It is the fourth issue for the same problem, other issues:
 - #1106262 (AIX 5, 2005)
 - #1234 (duplicate of #1106262, 2007)
 - #9700 (AIX 6)

The initial issue was related to semaphores and fork() (msg60639). Pass pshared=1 to sem_init() gets around this issue, but introduce new issues (msg60640). In general, locks and fork don't work together: there are still such issues in Python, see for example the issue #6721.

It looks like the behaviour of the semaphore depends on the underlying file system: see msg56228.

Nobody is able to get more information, or maybe nobody wants to dig this issue. It would be better to get the real cause of the issue, and maybe report the issue to the editor of AIX. I don't have access to this AIX, and only few Python developers have access to this OS.

At least, I would like to know if Sébastien Sablé (the author of the original patch) changed his opinion about this issue since 2007 ;-)
msg138962 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-06-24 16:57
> Nobody is able to get more information, or maybe nobody wants to dig this issue. It would be better to get the real cause of the issue,

Well, it looks like a libc or kernel bug, and we probably can't do
much about it. Just falling back to mutex + condition variable instead
of POSIX semaphores does the trick, so I'll say just avoid POSIX
semaphores altogether on AIX and we should be fine. People could
report this to their AIX vendor, but they should definitely consider
dropping this terminally broken platform and switch to, let's says,
Linux or BSD...
msg140606 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-07-18 17:20
So, what do we do now?
msg140653 - (view) Author: Sébastien Sablé (sable) Date: 2011-07-19 09:47
Hi,

sorry for the late reply.

 > STINNER Victor added the comment:
 >
> At least, I would like to know if Sébastien Sablé (the author of the original patch) changed his opinion about this issue since 2007 ;-)

I haven't changed my mind concerning this patch and we still use it for 
the Python that we deploy on AIX 5.3 and AIX 6.1 systems (I don't use 
AIX 7 yet).

Not using this patch will result in performance degradation and 
"sem_post: Permission denied" error messages from the system popping on 
the console.

 > Charles-François Natali added the comment:
 >
 > Well, it looks like a libc or kernel bug, and we probably can't do
 > much about it. Just falling back to mutex + condition variable instead
 > of POSIX semaphores does the trick, so I'll say just avoid POSIX
 > semaphores altogether on AIX and we should be fine. People could
 > report this to their AIX vendor, but they should definitely consider
 > dropping this terminally broken platform and switch to, let's says,
 > Linux or BSD...

I agree with Charles-François that it is an AIX bug that should be 
reported to IBM so that they could fix it in a future release.
I already submitted some "tickets" to IBM concerning other problems and 
it is not exactly an "agile" company so you have to be very patient.

I also agree that people should consider dropping this broken platform, 
but unfortunately IBM is aggressively and successfully (AFAIK) pushing 
this platform to many customers (in the finance industry in particular) 
so I don't expect it to die soon.

regards
msg140706 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-07-19 22:53
The patch looks fine to me.

> in the finance industry in particular

Ha, I've seen that. There are even proprietary desktop applications for AIX!
msg140820 - (view) Author: Roundup Robot (python-dev) Date: 2011-07-21 17:41
New changeset f5a7d413638d by Charles-François Natali in branch '2.7':
Issue #12372: POSIX semaphores are broken on AIX: don't use them.
http://hg.python.org/cpython/rev/f5a7d413638d
msg140821 - (view) Author: Roundup Robot (python-dev) Date: 2011-07-21 17:45
New changeset f0475f78d45c by Charles-François Natali in branch '3.2':
Issue #12372: POSIX semaphores are broken on AIX: don't use them.
http://hg.python.org/cpython/rev/f0475f78d45c
msg140822 - (view) Author: Roundup Robot (python-dev) Date: 2011-07-21 17:49
New changeset 44a02d6b74e4 by Charles-François Natali in branch 'default':
Merge - Issue #12372: POSIX semaphores are broken on AIX: don't use them.
http://hg.python.org/cpython/rev/44a02d6b74e4
msg140823 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-07-21 18:01
Patch committed: this should fix POSIX semaphores failures on AIX.
Reshmi, thanks for reporting this!
History
Date User Action Args
2011-07-21 18:01:45neologixsetstatus: open -> closed
resolution: fixed
messages: + msg140823

stage: commit review -> resolved
2011-07-21 17:49:43python-devsetmessages: + msg140822
2011-07-21 17:45:37python-devsetmessages: + msg140821
2011-07-21 17:41:08python-devsetnosy: + python-dev
messages: + msg140820
2011-07-19 22:53:02pitrousettype: behavior
stage: patch review -> commit review
messages: + msg140706
versions: + Python 3.2, Python 3.3, - Python 2.6, Python 3.1
2011-07-19 16:51:10neologixsetnosy: + pitrou
2011-07-19 09:47:08sablesetmessages: + msg140653
2011-07-18 17:20:23neologixsetmessages: + msg140606
2011-06-24 16:57:17neologixsetmessages: + msg138962
2011-06-23 21:25:38hayposetnosy: + sable
messages: + msg138873
2011-06-23 16:58:05neologixsetfiles: + aix_broken_sem.diff

nosy: + haypo, neologix
messages: + msg138871

keywords: + patch, needs review
stage: patch review
2011-06-20 08:56:08reshmi.georgecreate