classification
Title: BETA report: Python-3.6 build messages to stderr: AIX and "not GCC"
Type: Stage: resolved
Components: Build Versions: Python 3.6
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: David.Edelsohn, Michael.Felt, christian.heimes, martin.panter
Priority: normal Keywords:

Created on 2016-09-27 18:35 by Michael.Felt, last changed 2020-02-03 14:02 by Michael.Felt. This issue is now closed.

Files
File name Uploaded Description Edit
build-python-3.6.0.b1.script Michael.Felt, 2016-09-27 18:35 the stderr capture to the 'script' command during build of Python3-6.0b1, AIX xlc compiler
Messages (7)
msg277538 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-09-27 18:35
GCC and IBM xlC compilers pay attention to different things - so some of the messages are "accepted" if not harmless.

e.g. this type:
"./Modules/xxsubtype.c", line 293.19: 1506-196 (W) Initialization between types "void*" and "int(*)(struct _object*)" is not allowed.

Others are (well)-known different compiler behaviors - such as the default sign of a 'bit-field'
"/data/prj/python/python-3.6.0.177/Modules/_ctypes/_ctypes_test.c", line 389.5: 1506-159 (E) Bit field type specified for M is not valid. Type unsigned assumed.

However, some of the messages may lead to a problem down the road. Might be the platform, might be the "converted" include files:

"./pyconfig.h", line 1448.9: 1506-236 (W) Macro name _POSIX_C_SOURCE has been redefined.
"./pyconfig.h", line 1448.9: 1506-358 (I) "_POSIX_C_SOURCE" is defined on line 147 of /usr/include/standards.h.
"./pyconfig.h", line 1460.9: 1506-236 (W) Macro name _XOPEN_SOURCE has been redefined.
"./pyconfig.h", line 1460.9: 1506-358 (I) "_XOPEN_SOURCE" is defined on line 143 of /usr/include/standards.h.

And others might be more of an issue - e.g., no idea what "_blake2" is - does not look 'happy' or safe:
"/data/prj/python/python-3.6.0.177/Modules/_blake2/impl/blake2.h", line 89.14: 1506-508 (W) Option push for pragma pack is not supported.
"/data/prj/python/python-3.6.0.177/Modules/_blake2/impl/blake2.h", line 89.1: 1506-224 (W) Incorrect pragma ignored.
"/data/prj/python/python-3.6.0.177/Modules/_blake2/impl/blake2.h", line 119.14: 1506-726 (W) Attempting to pop an empty pack stack. Current pack may change.

That pretty well summarizes what is in the file - stdout was directed to a file
msg277542 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-09-27 19:14
blake2 is a hash algorithm. It uses #pragma pack(push, 1) and #pragma pack(pop) to change packing rules for structs.
msg278689 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-10-15 03:18
For the blake problem, I guess the structures may either get laid out incorrectly, or you might be lucky and they will get the desired packing by default. You might have to find if XLC has another way to enable struct packing. Or in the worst case, rewrite the code to pack data in a portable manner without using a struct.

The feature test macros like _POSIX_C_SOURCE are also discussed in various bug reports, including Issue 17120. How are you invoking /usr/include/standards.h? Perhaps there is a compiler mode or buggy header file that is getting in the way?

Regarding the data and function pointer confusion, this is a tricky case. I understand standard C (C99 etc) does not require function pointers to be compatible with void pointers. However, POSIX might require partial compatibility, at least for casting the void pointer from dlsym() to a function pointer: <http://pubs.opengroup.org/onlinepubs/9699919799/functions/dlsym.html#tag_16_96_07>.

Does AIX actually have incompatible (e.g. different size) function and void pointers, or can we safely ignore those warnings? I think it may be annoying to change Python’s PyType_Slot and PyModuleDef_Slot structures to be compatible with function pointers, though perhaps it could be done with anonymous unions or something.
msg278764 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-10-16 10:29
I'm cautious about messing with crypto code. The problem should be fixed upstream. Please talk the blake2 issue to https://github.com/BLAKE2/BLAKE2.
msg279275 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-10-23 19:26
re: the blake issue - I have ignored it as in not run any tests. Assistance would be to tell me how to easily test "blake" working or not working - before I head upstream.

re: _POSIX_C_SOURCE and _XOPEN_SOURCE

This is the logic in standards.h (AIX 5.3)
  +142  #if (!defined (_XOPEN_SOURCE)) &&  (!defined (_POSIX_SOURCE)) && (!defined (_ANSI_C_SOURCE))
  +143  #define _XOPEN_SOURCE  600
  +144  #define _XOPEN_SOURCE_EXTENDED 1
  +145  #define _POSIX_SOURCE
  +146  #ifndef _POSIX_C_SOURCE
  +147  #define _POSIX_C_SOURCE 200112L
  +148  #endif

And the same in AIX 6.1 (just different line numbers)
  +151  #if (!defined (_XOPEN_SOURCE)) &&  (!defined (_POSIX_SOURCE)) && (!defined (_ANSI_C_SOURCE))
  +152  #define _XOPEN_SOURCE  600
  +153  #define _XOPEN_SOURCE_EXTENDED 1
  +154  #define _POSIX_SOURCE
  +155  #ifndef _POSIX_C_SOURCE
  +156  #define _POSIX_C_SOURCE 200112L
  +157  #endif

However, AIX 7.1 has the "new" values:

  +160  #if (!defined (_XOPEN_SOURCE)) &&  (!defined (_POSIX_SOURCE)) && (!defined (_ANSI_C_SOURCE))
  +161  #define _XOPEN_SOURCE  700
  +162  #define _XOPEN_SOURCE_EXTENDED 1
  +163  #define _POSIX_SOURCE
  +164  #ifndef _POSIX_C_SOURCE
  +165  #define _POSIX_C_SOURCE 200809L
  +166  #endif

So, getting back to issue #17120 - and impact for AIX (and other older *nix platforms - is Python3 saying no support?)

As to the question about how the compiler is called ... Using, as much as possible - defaults documented below.

I use the invocation "xlc" generally - although I am thinking about changing to the "thread safe" invocation "xlc_r".

The differences are defined in /etc/vac.cfg.{53|61|71} (and I expect .72 for the latest compiler).

* -qlanglvl=extc99 C compiler with common extensions, UNIX headers
xlc:    use        = DEFLT_C
        crt        = /lib/crt0.o
        mcrt       = /lib/mcrt0.o
        gcrt       = /lib/gcrt0.o
        libraries  = -L/usr/vac/lib,-lxlopt,-lxlipa,-lxl,-lc
        proflibs   = -L/lib/profiled,-L/usr/lib/profiled
        options    = -qlanglvl=extc99,-qcpluscmt,-qkeyword=inline,-qalias=ansi


* standard c compiler aliased as xlc_r (53 Threads)
xlc_r:  use        = DEFLT_C
        crt        = /lib/crt0.o
        mcrt       = /lib/mcrt0.o
        gcrt       = /lib/gcrt0.o
        libraries  = -L/usr/vac/lib,-lxlopt,-lxlipa,-lxl,-lpthreads,-lc
        proflibs   = -L/lib/profiled,-L/usr/lib/profiled
        hdlibs     = -L/usr/vac/lib,-lhmd
        options    = -qlanglvl=extc99,-qcpluscmt,-qkeyword=inline,-qalias=ansi,-qthreaded,-D_THREAD_SAFE,-D__VACPP_MULTI__

As both have -qalias=ansi - that obviously does not define any of:
 +142  #if (!defined (_XOPEN_SOURCE)) &&  (!defined (_POSIX_SOURCE)) && (!defined (_ANSI_C_SOURCE))

FYI: the other defaults are here:

DEFLT_C:
        use           =DEFLT
        xlurt_cfg_path=/usr/vac/urt
        xlurt_cfg_name=urt_client.cfg

DEFLT_CPP:
        use           =DEFLT
        xlurt_cfg_path=/usr/vacpp/urt
        xlurt_cfg_name=urt_client.cfg

DEFLT:
        cppcomp   = /usr/vacpp/exe/xlCentry
        ccomp     = /usr/vac/exe/xlcentry
        code      = /usr/vac/exe/xlCcode
        cpp       = /usr/vac/exe/xlCcpp
        munch     = /usr/vacpp/exe/munch
        dis       = /usr/vac/exe/dis
        xlC       = /usr/vac/bin/xlc
        list      = /usr/vac/exe/xllist
        xslt      = /usr/vac/exe/XALAN
        transforms = /usr/vac/listings
        listlibs  = /usr/vac/lib
        cppinc    = /usr/vacpp/include
        ipa       = /usr/vac/exe/ipa
        cppfilt   = /usr/vacpp/bin/c++filt
        bolt      = /usr/vac/exe/bolt
        as        = /bin/as
        ld        = /bin/ld
        artool    = /bin/ar
        options   = -D_AIX,-D_AIX32,-D_AIX41,-D_AIX43,-D_AIX50,-D_AIX51,-D_AIX52,-D_AIX53,-D_IBMR2,-D_POWER
        options32 = -bpT:0x10000000,-bpD:0x20000000
        options32_bmaxdata = -bpT:0x10000000,-bpD:0x30000000
        options64 = -bpT:0x100000000,-bpD:0x110000000
        ldopt     = "b:o:e:u:R:H:Y:Z:L:T:A:k:j:"
        hdlibs    = -L/usr/vac/lib,-lhmd
        xlCcopt   = -qlanglvl=extc99,-qcpluscmt,-qkeyword=inline,-qalias=ansi
        crt_64    = /lib/crt0_64.o
        mcrt_64   = /lib/mcrt0_64.o
        gcrt_64   = /lib/gcrt0_64.o
        smplibraries = -lxlsmp
        palibraries  = -L/usr/vatools/lib,-lpahooks
        resexp     = /usr/vacpp/lib/res.exp
        genexports = /usr/vac/bin/CreateExportList
        vac_path   = /usr/vac
        vacpp_path = /usr/vacpp
        xlcmp_path = /usr/vac:/usr/vacpp
        xlc_c_stdinc   = /usr/vac/include:/usr/include
        xlc_cpp_stdinc = /usr/vacpp/include:/usr/include
        xlurt_msg_cat_name=vacumsg.cat
        __GNUC_MINOR__ = 3
        __GNUC_PATCHLEVEL__ = 4
        __GNUC__ = 3
        dfplibs = -ldecNumber
        oslevel   = 5.3
        os_variant = aix
        os_major  = 5
        os_minor  = 3
        inst_info = 5.3, 03 10 2015 16:09

I think the key "extra" is the default defines:
        options   = -D_AIX,-D_AIX32,-D_AIX41,-D_AIX43,-D_AIX50,-D_AIX51,-D_AIX52,-D_AIX53,-D_IBMR2,-D_POWER

Lastly,
re: pointer sizes - my expectation is that a pointer size is a pointer size (and that size being dependent on ABI 4 bytes, or 8 bytes) - but I will have to find or write again my little "sizeof" program. Will do that later. I think IBM "lint" is just being very strict. However, I have learned with working on a port of dovecat that gcc and xlc have different "rules" aka permissiveness when it comes to "dynamic" structs.
msg309337 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-01-01 17:19
this was just feedback - and should probably be closed.
msg361293 - (view) Author: Michael Felt (Michael.Felt) * Date: 2020-02-03 14:02
Closing, as not longer relevant.
History
Date User Action Args
2020-02-03 14:02:18Michael.Feltsetstatus: open -> closed

messages: + msg361293
stage: resolved
2018-01-02 15:25:00David.Edelsohnsetnosy: + David.Edelsohn
2018-01-01 17:19:31Michael.Feltsetmessages: + msg309337
2016-10-23 19:26:35Michael.Feltsetmessages: + msg279275
2016-10-16 10:29:48christian.heimessetmessages: + msg278764
2016-10-15 03:18:49martin.pantersetnosy: + martin.panter
messages: + msg278689
2016-10-15 02:22:20martin.panterlinkissue27844 superseder
2016-09-27 19:14:20christian.heimessetnosy: + christian.heimes
messages: + msg277542
2016-09-27 18:35:35Michael.Feltsetversions: + Python 3.6
2016-09-27 18:35:30Michael.Feltcreate