classification
Title: AIX posix_fadvise and posix_fallocate
Type: behavior Stage:
Components: Tests Versions: Python 3.5, Python 3.3, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: David.Edelsohn, pitrou, python-dev, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2014-09-12 18:20 by David.Edelsohn, last changed 2014-09-30 10:41 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
10812_aix.patch David.Edelsohn, 2014-09-12 18:20 review
22396_aix.patch David.Edelsohn, 2014-09-23 13:25 review
Messages (13)
msg226834 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2014-09-12 18:20
As with Solaris and Issue10812, test_posix fadvise and fallocate fail on AIX.  Python is compiled with _LARGE_FILES, which changes the function signature for posix_fadvise and posix_fallocate so that off_t is "long long" on 32 bit system passed in two registers.  The Python call to those functions does not place the arguments in the correct registers, causing an EINVAL error.  This patch fixes the failures in a similar way to Solaris ZFS kludge for Issue10812.
msg226836 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-12 18:24
> The Python call to those functions does not place the arguments in the correct registers

Well... isn't there a way to fix this? I don't understand how this issue can come up.
msg226838 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-12 18:44
See similar Ruby issue: https://bugs.ruby-lang.org/issues/9914 .

As workaround we can redeclare posix_fadvise as "int posix_fadvise(int fd, long offset, long len, int advice)" on 32-bit AIX with enabled _LARGE_FILES. More safe option is to disable posix_fadvise in such case (as Ruby had done).
msg227293 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2014-09-22 18:58
Any feedback about which approach would be acceptable?
msg227316 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-22 22:29
10812_aix.patch just hides the problem.

I understand that AIX doesn't declare the function prototype correctly? I would prefer to disable the function in the posix module (don't declare it) if it's the case.
msg227317 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-22 22:35
> I understand that AIX doesn't declare the function prototype correctly?

AIX bug report:
http://www-01.ibm.com/support/docview.wss?uid=isg1IV56170

I like Ruby's patch:

-#ifdef HAVE_POSIX_FADVISE
+    /* AIX currently does not support a 32-bit call to posix_fadvise()
+     * if _LARGE_FILES is defined.
+     */
+#if defined(HAVE_POSIX_FADVISE) && !(defined(_AIX) && defined(_LARGE_FILES) && !defined(_ARCH_PPC64))
msg227355 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2014-09-23 13:25
Attached is a revised patch that disables posix_fadvise() and posix_fallocate() when building on 32 bit AIX with _LARGE_FILES defined.
msg227356 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-23 13:32
> Attached is a revised patch that disables posix_fadvise() and posix_fallocate() when building on 32 bit AIX with _LARGE_FILES defined.

Good. You should add a reference to this issue, something like "Issue #22396: ...".

To avoid code duplication, you may write something like:

/* Issue #22396: AIX currently does not support a 32-bit 
   call to posix_fallocate() if _LARGE_FILES is defined. */
#if defined(HAVE_POSIX_FALLOCATE) && !(defined(_AIX) && defined(_LARGE_FILES) && !defined(__64BIT__))
#  undef HAVE_POSIX_FALLOCATE
#endif

or "#define BROKEN_POSIX_FALLOCATE".

Which Python versions should be patched? 3.4 and 3.5? Python 2.7 doesn't have the function (introduced in Python 3.3). For Python 3.4, it means that between two Python minor versions, the function disappears on AIX 32-bit :-/ Is it a problem since the function didn't work on this platform? (always fail with EINVAL)

I suggest to patch 3.4 and 3.5.
msg227357 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-23 13:44
I think that some time AIX bug will be fixed. May be it would be better to introduce special macros, and set it in the configure script (similar to HAVE_GLIBC_MEMMOVE_BUG or HAVE_IPA_PURE_CONST_BUG). Or may be just udefine HAVE_POSIX_FADVISE at such circumstances.
msg227358 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-23 13:48
Or can we simply keep the function and skip the test?
msg227359 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2014-09-23 13:56
The declaration of the two system calls should be fixed in the AIX header, but the clueless response to the AIX problem report is underwhelming.

I don't understand the "keep the function and skip the test" suggestion.  I thought that was my first patch -- catch the exception of invalid argument and allow it to fail on AIX.  If AIX eventually is fixed, the test will pass, no harm, no foul.
msg227875 - (view) Author: Roundup Robot (python-dev) Date: 2014-09-30 10:36
New changeset 8e5e19b3cd4e by Victor Stinner in branch '3.4':
Issue #22396: On 32-bit AIX platform, don't expose os.posix_fadvise() nor
https://hg.python.org/cpython/rev/8e5e19b3cd4e

New changeset 5ade1061fa3d by Victor Stinner in branch 'default':
(Merge 3.4) Issue #22396: On 32-bit AIX platform, don't expose
https://hg.python.org/cpython/rev/5ade1061fa3d
msg227876 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-30 10:41
> Or can we simply keep the function and skip the test?

Usually, we prefer to not provide the function in Python if it is known to be broken. Other examples:

- HAVE_BROKEN_POLL: don't declare select.poll()
- HAVE_BROKEN_PTHREAD_SIGMASK: don't declare signal.pthread_sigmask()

There are also HAVE_BROKEN_NICE and HAVE_BROKEN_UNSETENV.

Sorry, I'm too lazy to hack configure.ac, so I used a simple #ifdef in Modules/posixmodule.c to define "POSIX_FADVISE_AIX_BUG". If you feel more confortable with autotools, don't hesitate to propose a better patch :-)

I consider that the issue is now fixed.

@David: You should check with IBM why they don't fix the issue.
History
Date User Action Args
2014-09-30 10:41:09vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg227876
2014-09-30 10:36:20python-devsetnosy: + python-dev
messages: + msg227875
2014-09-23 13:56:42David.Edelsohnsetmessages: + msg227359
2014-09-23 13:48:31pitrousetmessages: + msg227358
2014-09-23 13:44:07serhiy.storchakasetmessages: + msg227357
2014-09-23 13:32:57vstinnersetmessages: + msg227356
2014-09-23 13:25:57David.Edelsohnsetfiles: + 22396_aix.patch

messages: + msg227355
2014-09-22 22:35:01vstinnersetmessages: + msg227317
2014-09-22 22:29:25vstinnersetnosy: + vstinner
messages: + msg227316
2014-09-22 18:58:27David.Edelsohnsetmessages: + msg227293
2014-09-12 18:44:59serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg226838
2014-09-12 18:24:03pitrousetmessages: + msg226836
2014-09-12 18:20:03David.Edelsohncreate