classification
Title: Error build Python with Intel compiler: doesn't provide atomic_uintptr_t
Type: compile error Stage: resolved
Components: Build Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Glenn Johnson, Saszalez, cberger, miss-islington, rakeller, skrah, vstinner
Priority: normal Keywords: patch

Created on 2019-06-26 15:32 by Saszalez, last changed 2019-10-22 20:27 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
stdatomic.h Saszalez, 2019-06-27 21:51
python.patch rakeller, 2019-09-27 15:14
Pull Requests
URL Status Linked Edit
PR 16717 merged vstinner, 2019-10-11 13:37
PR 16892 merged miss-islington, 2019-10-22 19:54
PR 16893 merged miss-islington, 2019-10-22 19:54
Messages (17)
msg346637 - (view) Author: Borja (Saszalez) Date: 2019-06-26 15:32
I'm trying to build Python 3.7 on CentOS 7 with Intel compilers. To compile I do the following:

export CC=icc
export CXX=icpc
export FC=ifort

./configure --prefix=my_prefix --enable-optimizations --without-gcc
make
make install

In the configure, no error is seen, but when I execute make, the following message is printed:

Running code to generate profile data (this can take a while):
# First, we need to create a clean build with profile generation
# enabled.
make profile-gen-stamp
make[1]: se ingresa al directorio `/home/python/source/Python-3.7.3'
Building with support for profile generation:
make build_all_generate_profile
make[2]: se ingresa al directorio `/home/python/source/Python-3.7.3'
make build_all CFLAGS_NODIST=" -prof-gen" LDFLAGS_NODIST=" -prof-gen" LIBS="-lcrypt -lpthread -ldl  -lutil"
make[3]: se ingresa al directorio `/home/python/source/Python-3.7.3'
icc -pthread -c -Wsign-compare -Wunreachable-code -DNDEBUG -g  -O3 -Wall    -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wno-cast-function-type -Werror=implicit-function-declaration -fp-model strict -prof-gen  -I. -I./Include    -DPy_BUILD_CORE -o Programs/python.o ./Programs/python.c
icc: command line warning #10006: ignoring unknown option '-Wno-cast-function-type'
In file included from ./Include/Python.h(75),
                 from ./Programs/python.c(3):
./Include/pyatomic.h(39): error: identifier "atomic_uintptr_t" is undefined
      atomic_uintptr_t _value;
      ^

compilation aborted for ./Programs/python.c (code 2)
msg346639 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-26 15:42
atomic_uintptr_t is used if HAVE_STD_ATOMIC defined is defined. Can you please check in pyconfig.h if it's defined?

So <stdatomic.h> doesn't provide atomic_uintptr_t with ICC?
msg346716 - (view) Author: Borja (Saszalez) Date: 2019-06-27 08:07
In pyconfig.h the variable HAVE_STD_ATOMIC appears like this:

/* Has stdatomic.h with atomic_int */
#define HAVE_STD_ATOMIC 1

I searched at <stdatomic.h> atomic_uintptr_t and found nothing.
msg346739 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-27 14:25
> I searched at <stdatomic.h> atomic_uintptr_t and found nothing.

That's not good. I expectd std to stand for standard. I expected that atomic_uintptr_t would always be available on <stdatomic.h>.

GCC defines the type as:

typedef _Atomic __UINTPTR_TYPE__ atomic_uintptr_t;

Maybe you should try:

typedef struct _Py_atomic_address {
    _Atomic Py_uintptr_t _value;
} _Py_atomic_address;
msg346778 - (view) Author: Borja (Saszalez) Date: 2019-06-27 21:51
Two things.

I searched a bit more and found "atomic_uint" inside <stdatomic.h>, but it's not exactly the same as what you wrote.

What you have put me to try, where do I put it?

I attached <stdatomic.h> in case it helps.
msg346785 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-27 23:23
I suggested you to test the following change on the master branch of Python:

diff --git a/Include/internal/pycore_atomic.h b/Include/internal/pycore_atomic.h
index 336bc3fec2..c624a0cf1c 100644
--- a/Include/internal/pycore_atomic.h
+++ b/Include/internal/pycore_atomic.h
@@ -44,7 +44,7 @@ typedef enum _Py_memory_order {
 } _Py_memory_order;
 
 typedef struct _Py_atomic_address {
-    atomic_uintptr_t _value;
+    _Atomic uintptr_t _value;
 } _Py_atomic_address;
 
 typedef struct _Py_atomic_int {
msg348268 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2019-07-21 23:05
Is it available with -std=c11? It is a bit strange that we use -std=c99. I thought that header is C11:

https://en.cppreference.com/w/c/atomic
msg349081 - (view) Author: Christian Berger (cberger) Date: 2019-08-05 22:19
I have the same problem on RH6 with icc 2019.4 and Python 3.6.9. Do you want a new bug for that?
Changing Include/pyatomic.h to use _Atomic as suggested by Victor in msg346785 leads to the error that _Atomic was undefined.
msg349471 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-08-12 13:22
> Changing Include/pyatomic.h to use _Atomic as suggested by Victor in msg346785 leads to the error that _Atomic was undefined.

No idea how to define a "_Py_atomic_address" type with icc 2019.4 in this case. If someone has a working patch, I can review it ;-)
msg353373 - (view) Author: Rainer Keller (rakeller) Date: 2019-09-27 15:14
We have been experiencing the same problem with compiling Python-3.7.4 on a system with CentOS 7.6 (aka glib-2.17): Intel Compiler 2019.6 implements stdatomic.h. but lacks the definition of atomic_uintptr_t.

From a configure-point of view just failing to define HAVE_STDATOMIC_H would suffice.
Therefore just add another otherwise unused variable (which might warn & bail out in case of -Werror / -Werror-all)...

The following patch works for us to compile.

Granted, the "proper" way would be to add another configure check for the underlying type and not blurry the check for stdatomic.h -- but then again, Intel should just implement all of C11...
msg354452 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-11 13:38
I wrote PR 16717 which uses a similar approach than python.patch.
msg354453 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-11 13:46
Can someone please test PR 16717 with ICC?
msg355039 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-21 10:07
I marked bpo-35473 as duplicate of this issue.
msg355145 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-22 19:53
New changeset 028f7349a0f6eaea0fec31becb587fcdf6e3cb28 by Victor Stinner in branch 'master':
bpo-37415: Fix stdatomic.h header check for ICC compiler (GH-16717)
https://github.com/python/cpython/commit/028f7349a0f6eaea0fec31becb587fcdf6e3cb28
msg355148 - (view) Author: miss-islington (miss-islington) Date: 2019-10-22 20:17
New changeset b102e4f05278c1b06130885eba961bd0193733b4 by Miss Skeleton (bot) in branch '3.7':
bpo-37415: Fix stdatomic.h header check for ICC compiler (GH-16717)
https://github.com/python/cpython/commit/b102e4f05278c1b06130885eba961bd0193733b4
msg355149 - (view) Author: miss-islington (miss-islington) Date: 2019-10-22 20:19
New changeset dbcea39ba713cc5b31fa1a243b16a0128c231c98 by Miss Skeleton (bot) in branch '3.8':
bpo-37415: Fix stdatomic.h header check for ICC compiler (GH-16717)
https://github.com/python/cpython/commit/dbcea39ba713cc5b31fa1a243b16a0128c231c98
msg355152 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-22 20:27
According to Adam J. Stewart, my PR 16717 fix the Python build on ICC:
https://github.com/python/cpython/pull/16717#issuecomment-544812182

So I merged it in 3.7, 3.8 and master branches.

Adam saw other failures, but it's unrelated issues which should be reported separately.

Thanks for the bug report Borja. It should now be fixed. Until a new Python version is released with the fix, you should either apply the patch manually or use a different compiler. GCC and Clang are well supported on Linux for example.
History
Date User Action Args
2019-11-12 21:12:49vstinnerlinkissue37855 superseder
2019-10-22 20:27:23vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg355152

stage: patch review -> resolved
2019-10-22 20:19:35miss-islingtonsetmessages: + msg355149
2019-10-22 20:17:26miss-islingtonsetnosy: + miss-islington
messages: + msg355148
2019-10-22 19:54:37miss-islingtonsetpull_requests: + pull_request16432
2019-10-22 19:54:04miss-islingtonsetpull_requests: + pull_request16431
2019-10-22 19:53:55vstinnersetmessages: + msg355145
2019-10-21 10:07:38vstinnersetmessages: + msg355039
2019-10-21 10:07:24vstinnerlinkissue35473 superseder
2019-10-11 13:46:35vstinnersetmessages: + msg354453
2019-10-11 13:38:46vstinnersetmessages: + msg354452
components: + Build
versions: + Python 3.7, Python 3.8, Python 3.9
2019-10-11 13:37:51vstinnersetstage: patch review
pull_requests: + pull_request16294
2019-09-27 15:14:40rakellersetfiles: + python.patch

nosy: + rakeller
messages: + msg353373

keywords: + patch
2019-08-12 13:22:26vstinnersetmessages: + msg349471
2019-08-05 22:19:00cbergersetnosy: + cberger
messages: + msg349081
2019-07-21 23:05:43skrahsetnosy: + skrah
messages: + msg348268
2019-07-21 22:49:54Glenn Johnsonsetnosy: + Glenn Johnson
2019-06-27 23:23:02vstinnersetmessages: + msg346785
2019-06-27 21:51:32Saszalezsetfiles: + stdatomic.h

messages: + msg346778
2019-06-27 14:25:35vstinnersetmessages: + msg346739
title: Error build Python with Intel compiler -> Error build Python with Intel compiler: <stdatomic.h> doesn't provide atomic_uintptr_t
2019-06-27 08:07:42Saszalezsetmessages: + msg346716
2019-06-26 15:42:01vstinnersetnosy: + vstinner
messages: + msg346639
2019-06-26 15:32:47Saszalezcreate