Issue35198
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2018-11-09 15:20 by Ayappan, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
pip3_build_pandas_0001.text | Michael.Felt, 2018-11-10 15:41 | complete pip3 build pandas output |
Pull Requests | |||
---|---|---|---|
URL | Status | Linked | Edit |
PR 10437 | merged | kadler, 2018-11-09 17:34 | |
PR 12162 | merged | miss-islington, 2019-03-04 14:48 |
Messages (15) | |||
---|---|---|---|
msg329522 - (view) | Author: (Ayappan) | Date: 2018-11-09 15:20 | |
I am trying to build pandas-0.23.4 using python3 in AIX. Everything goes fine till it encounters a cpp file. Below is the compile error output. building 'pandas._libs.window' extension gcc -fPIC -maix64 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DAIX_GENUINE_CPLUSCPLUS -D_LINUX_SOURCE_COMPAT -Wl,-brtl -O2 -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DAIX_GENUINE_CPLUSCPLUS -D_LINUX_SOURCE_COMPAT -Wl,-brtl -O2 -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/opt/freeware/lib64/python3.6/site-packages/numpy/core/include -I/opt/freeware/include/python3.6m -c pandas/_libs/window.cpp -o build/temp.aix-7.2-3.6/pandas/_libs/window.o -Wno-unused-function In file included from /opt/freeware/lib64/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1821, from /opt/freeware/lib64/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:18, from /opt/freeware/lib64/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4, from pandas/_libs/window.cpp:587: /opt/freeware/lib64/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp] #warning "Using deprecated NumPy API, disable it by " \ ^~~~~~~ g++ gcc -maix64 -pthread -bI:/opt/freeware/lib/python3.6/config-3.6m/python.exp -L. -L/usr/include -L/opt/freeware/include -L/usr/lib/threads -L/opt/freeware/lib64 -L/opt/freeware/lib -L/usr/lib64 -L/usr/lib -L. -L/usr/include -L/opt/freeware/include -L/usr/lib/threads -L/opt/freeware/lib64 -L/opt/freeware/lib -L/usr/lib64 -L/usr/lib build/temp.aix-7.2-3.6/pandas/_libs/window.o -L/opt/freeware/lib64 -o build/lib.aix-7.2-3.6/pandas/_libs/window.so g++: error: gcc: No such file or directory g++: error: unrecognized command line option '-bI:/opt/freeware/lib/python3.6/config-3.6m/python.exp' error: command 'g++' failed with exit status 1 Seems like somewhere setting up the proper compile options for compiling cpp files is not proper. Any hint will be really useful. |
|||
msg329622 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2018-11-10 15:41 | |
There are, perhaps, other issues as well. After a build of "master" a) pip3 install pandas - failed to find/download and build numpy - after "manual" pip3 install numpy and got to " six, python-dateutil, pytz, pandas" Then: Running setup.py install for pandas ... error Complete output from command /opt/bin/python3.8 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-f5rvzoc0/pandas/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-kmi2p9sd/install-record.txt --single-version-externally-managed --compile: ... UPDATING build/lib.aix-6.1-3.8-pydebug/pandas/_version.py set build/lib.aix-6.1-3.8-pydebug/pandas/_version.py to '0.23.4' running build_ext building 'pandas._libs.algos' extension creating build/temp.aix-6.1-3.8-pydebug creating build/temp.aix-6.1-3.8-pydebug/pandas creating build/temp.aix-6.1-3.8-pydebug/pandas/_libs xlc_r -O -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/opt/lib/python3.8/site-packages/numpy/core/include -I/opt/include/python3.8dm -c pandas/_libs/algos.c -o build/temp.aix-6.1-3.8-pydebug/pandas/_libs/algos.o -Wno-unused-function xlc_r: 1501-210 (S) command option Wno-unused-function contains an incorrect subargument error: command 'xlc_r' failed with exit status 40 ---------------------------------------- Command "/opt/bin/python3.8 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-f5rvzoc0/pandas/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-kmi2p9sd/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-f5rvzoc0/pandas/ So, it looks like, using xlc - I may not have even gotten as far window.gcc due to issues in pandas (the incorrect flag -Wno-used-function - have to find where that came from!) |
|||
msg329625 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2018-11-10 16:17 | |
Getting farther - after "hacking" pandas setup.py to not force a gcc flag. Still not getting as far, I think, as the initial poster. ... /opt/include/python3.8dm -c pandas/_libs/parsers.c -o build/temp.aix-6.1-3.8-pydebug/pandas/_libs/parsers.o xlc_r -O -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/opt/lib/python3.8/site-packages/numpy/core/include -I/opt/include/python3.8dm -c pandas/_libs/src/parser/tokenizer.c -o build/temp.aix-6.1-3.8-pydebug/pandas/_libs/src/parser/tokenizer.o "/opt/include/python3.8dm/pyconfig.h", line 1538.9: 1506-236 (W) Macro name _POSIX_C_SOURCE has been redefined. "/opt/include/python3.8dm/pyconfig.h", line 1538.9: 1506-358 (I) "_POSIX_C_SOURCE" is defined on line 160 of /usr/include/standards.h. "/opt/include/python3.8dm/pyconfig.h", line 1553.9: 1506-236 (W) Macro name _XOPEN_SOURCE has been redefined. "/opt/include/python3.8dm/pyconfig.h", line 1553.9: 1506-358 (I) "_XOPEN_SOURCE" is defined on line 156 of /usr/include/standards.h. "/usr/include/unistd.h", line 171.17: 1506-343 (S) Redeclaration of lseek64 differs from previous declaration on line 169 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 171.17: 1506-050 (I) Return type "long long" in redeclaration is not compatible with the previous return type "long". "/usr/include/unistd.h", line 171.17: 1506-377 (I) The type "long long" of parameter 2 differs from the previous type "long". "/usr/include/sys/lockf.h", line 64.20: 1506-343 (S) Redeclaration of lockf64 differs from previous declaration on line 62 of "/usr/include/sys/lockf.h". "/usr/include/sys/lockf.h", line 64.20: 1506-377 (I) The type "long long" of parameter 3 differs from the previous type "long". "/usr/include/unistd.h", line 809.33: 1506-343 (S) Redeclaration of ftruncate64 differs from previous declaration on line 807 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 809.33: 1506-377 (I) The type "long long" of parameter 2 differs from the previous type "long". "/usr/include/unistd.h", line 845.33: 1506-343 (S) Redeclaration of truncate64 differs from previous declaration on line 843 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 845.33: 1506-377 (I) The type "long long" of parameter 2 differs from the previous type "long". "/usr/include/unistd.h", line 862.33: 1506-343 (S) Redeclaration of pread64 differs from previous declaration on line 859 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 862.33: 1506-377 (I) The type "long long" of parameter 4 differs from the previous type "long". "/usr/include/unistd.h", line 863.33: 1506-343 (S) Redeclaration of pwrite64 differs from previous declaration on line 860 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 863.33: 1506-377 (I) The type "long long" of parameter 4 differs from the previous type "long". "/usr/include/unistd.h", line 942.25: 1506-343 (S) Redeclaration of fclear64 differs from previous declaration on line 939 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 942.25: 1506-050 (I) Return type "long long" in redeclaration is not compatible with the previous return type "long". "/usr/include/unistd.h", line 942.25: 1506-377 (I) The type "long long" of parameter 2 differs from the previous type "long". "/usr/include/unistd.h", line 943.25: 1506-343 (S) Redeclaration of fsync_range64 differs from previous declaration on line 940 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 943.25: 1506-377 (I) The type "long long" of parameter 3 differs from the previous type "long". "pandas/_libs/src/parser/tokenizer.c", line 265.40: 1506-280 (W) Function argument assignment between types "long long*" and "unsigned long*" is not allowed. "pandas/_libs/src/parser/tokenizer.c", line 292.30: 1506-280 (W) Function argument assignment between types "long long*" and "unsigned long*" is not allowed. "pandas/_libs/src/parser/tokenizer.c", line 323.28: 1506-280 (W) Function argument assignment between types "long long*" and "unsigned long*" is not allowed. error: command 'xlc_r' failed with exit status 1 root@x066:[/data/prj/python/pandas-0.23.4]set | grep OBJ root@x066:[/data/prj/python/pandas-0.23.4]python3 Python 3.8.0a0 (heads/master-dirty:b9498e2367, Nov 10 2018, 13:51:41) [C] on aix6 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.maxsize() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'int' object is not callable >>> sys.maxsize 2147483647 +++++ Will uninstall and build a 64-bit python3 and see if that helps. +++++ |
|||
msg329684 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2018-11-11 20:10 | |
Still getting the same errors, even with 64-bit build, so still not close to testing the actual problem. FYI: last time I build pandas the version was 0.19.0 - then it was 'simple'. Anyway, short of a simple (test) python module that includes c++ code, I'll look at changing the pandas build files to force any c++ file to the front. More - (asap, but still when I have time) |
|||
msg329722 - (view) | Author: (Ayappan) | Date: 2018-11-12 07:17 | |
The PR 10437 worked !!!. Now i am able to compile pandas-0.23.4 Michael Felt, About the re-declaration issues you are facing, that is because of the "#define _LARGE_FILES 1" in pyconfig-ppc64.h file. For 64bit build we actually don't need it. For 32bit build, we need it and also we have to pass the flag "-D_LARGE_FILES" through the _sysconfigdata_m_aix6_.py , otherwise one will get this re-declaration error. |
|||
msg332961 - (view) | Author: Kevin (kadler) * | Date: 2019-01-04 05:31 | |
Just a friendly ping that there's a PR for this bug waiting to be reviewed. |
|||
msg332971 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2019-01-04 11:54 | |
While the PR probably solves this - there is a 'bug' in pandas (I expect) that prevents me from completing the test - as, I expect LONG before the .cpp source is to be compiled - there is a error because a wrong flag is passed to the compiler (-Wno-unused-function) and the build stops. Have not had the time to disect pandas so that I can get a .cpp source compiled. +++++ Successfully installed pip-18.1 root@x066:[/data/prj/python/git/kadler]pip list /opt/lib/python3.8/site-packages/pip/_vendor/html5lib/_trie/_base.py:3: DeprecationWdeprecated, and in 3.8 it will stop working from collections import Mapping Package Version ---------- ------- pip 18.1 setuptools 39.0.1 root@x066:[/data/prj/python/git/kadler]pip install numpy /opt/lib/python3.8/site-packages/pip/_vendor/html5lib/_trie/_base.py:3: DeprecationWdeprecated, and in 3.8 it will stop working from collections import Mapping Collecting numpy /opt/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py:220: PendingDepreca warnings.warn( Using cached https://files.pythonhosted.org/packages/2d/80/1809de155bad674b494248b Installing collected packages: numpy Running setup.py install for numpy ... done Successfully installed numpy-1.15.4 ... Installing collected packages: six, python-dateutil, pytz, pandas ... copying pandas/io/formats/templates/html.tpl -> build/lib.aix-6.1-3.8-pydebug/pandas/io/formats/templates UPDATING build/lib.aix-6.1-3.8-pydebug/pandas/_version.py set build/lib.aix-6.1-3.8-pydebug/pandas/_version.py to '0.23.4' running build_ext building 'pandas._libs.algos' extension creating build/temp.aix-6.1-3.8-pydebug creating build/temp.aix-6.1-3.8-pydebug/pandas creating build/temp.aix-6.1-3.8-pydebug/pandas/_libs xlc_r -O -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/opt/lib/python3.8/site-packages/numpy/core/include -I/opt/include/python3.8dm -c pandas/_libs/algos.c -o build/temp.aix-6.1-3.8-pydebug/pandas/_libs/algos.o -Wno-unused-function xlc_r: 1501-210 (S) command option Wno-unused-function contains an incorrect subargument error: command 'xlc_r' failed with exit status 40 ---------------------------------------- Command "/opt/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-xnscgehv/pandas/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-0w46p413/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-xnscgehv/pandas/ |
|||
msg332984 - (view) | Author: Kevin (kadler) * | Date: 2019-01-04 16:08 | |
Ah. We always compile with GCC, so would not have hit that particular problem. |
|||
msg332989 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2019-01-04 18:37 | |
On 04/01/2019 17:08, Kevin wrote: > Kevin <kadler@us.ibm.com> added the comment: > > Ah. We always compile with GCC, so would not have hit that particular problem. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <https://bugs.python.org/issue35198> > _______________________________________ > FYI. I found a 'hack' in ./setup.py to skip adding the argument. Progressing. |
|||
msg332991 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2019-01-04 19:04 | |
Further along - however, I never get to the "link" routine. Again, this is likely a pandas coding issue - currently python is calling xlc_r ..., but when I manually modify it to xlC_r I get the same error. xlc_r -D_LARGE_FILES -O -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -Ipandas/_libs -I./pandas/_libs -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/opt/lib/python3.8/site-packages/numpy/core/include -I/opt/include/python3.8dm -c pandas/_libs/window.cpp -o build/temp.aix-6.1-3.8-pydebug/pandas/_libs/window.o "pandas/_libs/window.cpp", line 9943.23: 1540-0063 (S) The text "(" is unexpected. "pandas/_libs/window.cpp", line 10030.23: 1540-0063 (S) The text "(" is unexpected. "pandas/_libs/window.cpp", line 11165.21: 1540-0063 (S) The text "(" is unexpected. error: command 'xlc_r' failed with exit status 1 root@x066:[/data/prj/aixtools/git/pandas-master]ebug/pandas/_libs/window.o < "pandas/_libs/window.cpp", line 9943.23: 1540-0063 (S) The text "(" is unexpected. "pandas/_libs/window.cpp", line 10030.23: 1540-0063 (S) The text "(" is unexpected. "pandas/_libs/window.cpp", line 11165.21: 1540-0063 (S) The text "(" is unexpected. root@x066:[/data/prj/aixtools/git/pandas-master]xlC_r -D_LARGE_FILES -O -I/opt/include -O2 -qm> "pandas/_libs/window.cpp", line 9943.23: 1540-0063 (S) The text "(" is unexpected. "pandas/_libs/window.cpp", line 10030.23: 1540-0063 (S) The text "(" is unexpected. "pandas/_libs/window.cpp", line 11165.21: 1540-0063 (S) The text "(" is unexpected. FYI. Will look more tomorrow, but if you have an idea, like the -D_LARGE_FILES fix, I am much obliged. |
|||
msg333231 - (view) | Author: (Ayappan) | Date: 2019-01-08 14:40 | |
Not sure what went wrong here. I used gcc & g++ and didn't hit this issue. |
|||
msg333342 - (view) | Author: Michael Felt (Michael.Felt) * | Date: 2019-01-09 19:18 | |
On 08/01/2019 15:40, Ayappan wrote: > Ayappan <ayappanec@gmail.com> added the comment: > > Not sure what went wrong here. > I used gcc & g++ and didn't hit this issue. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <https://bugs.python.org/issue35198> > _______________________________________ > Well, it is probably an issue with xlc as I have vac.C and vacpp.cmp.core installed (aka xlc and xlC). These days it seems only gcc environment gets tested in depth and portability issues with "posix" compilers get missed. GCC is a fine compiler - just not the one I am using for a number of "logistical" reasons. Thanks for the update! Michael |
|||
msg337123 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-03-04 14:48 | |
New changeset 800d5cd75025876d79ab05980925a05d8e36b63d by Victor Stinner (Kevin Adler) in branch 'master': bpo-35198 Fix C++ extension compilation on AIX (GH-10437) https://github.com/python/cpython/commit/800d5cd75025876d79ab05980925a05d8e36b63d |
|||
msg337128 - (view) | Author: miss-islington (miss-islington) | Date: 2019-03-04 15:06 | |
New changeset 06e9953d5e3f0144ec8247b61541e7be85d55b50 by Miss Islington (bot) in branch '3.7': bpo-35198 Fix C++ extension compilation on AIX (GH-10437) https://github.com/python/cpython/commit/06e9953d5e3f0144ec8247b61541e7be85d55b50 |
|||
msg337130 - (view) | Author: STINNER Victor (vstinner) * | Date: 2019-03-04 15:25 | |
Thanks Kevin Adler. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:07 | admin | set | github: 79379 |
2019-03-04 15:25:03 | vstinner | set | status: open -> closed resolution: fixed messages: + msg337130 stage: patch review -> resolved |
2019-03-04 15:06:39 | miss-islington | set | nosy:
+ miss-islington messages: + msg337128 |
2019-03-04 14:48:54 | miss-islington | set | pull_requests: + pull_request12160 |
2019-03-04 14:48:46 | vstinner | set | nosy:
+ vstinner messages: + msg337123 |
2019-01-09 19:18:38 | Michael.Felt | set | messages: + msg333342 |
2019-01-08 14:40:04 | Ayappan | set | messages: + msg333231 |
2019-01-04 19:04:33 | Michael.Felt | set | messages: + msg332991 |
2019-01-04 18:37:32 | Michael.Felt | set | messages: + msg332989 |
2019-01-04 16:08:22 | kadler | set | messages: + msg332984 |
2019-01-04 11:54:42 | Michael.Felt | set | messages: + msg332971 |
2019-01-04 05:31:44 | kadler | set | nosy:
+ kadler messages: + msg332961 |
2018-11-12 07:17:05 | Ayappan | set | messages: + msg329722 |
2018-11-11 20:10:55 | Michael.Felt | set | messages: + msg329684 |
2018-11-10 16:17:42 | Michael.Felt | set | versions: + Python 3.8 |
2018-11-10 16:17:23 | Michael.Felt | set | messages: + msg329625 |
2018-11-10 15:41:08 | Michael.Felt | set | files:
+ pip3_build_pandas_0001.text messages: + msg329622 |
2018-11-09 19:00:38 | ned.deily | set | nosy:
+ David.Edelsohn, Michael.Felt |
2018-11-09 17:34:33 | kadler | set | keywords:
+ patch stage: patch review pull_requests: + pull_request9711 |
2018-11-09 15:20:45 | Ayappan | create |