msg414809 - (view) |
Author: Oleksandr Pavlyk (oleksandr-pavlyk) * |
Date: 2022-03-09 19:07 |
The following snippet illustrates request by an extension to use AMX registers:
```
// no increase sigaltstack size fix will not wark till we fix python
void enable_amx_no_fix()
{
unsigned long bitmask;
long rc;
rc = syscall(SYS_arch_prctl, ARCH_REQ_XCOMP_PERM, XFEATURE_XTILEDATA);
if (rc) {
printf("The kernel rejects the AMX use.\n");
printf("errno %d\n",errno);
} else {
printf("The kernel allows to use AMX.\n");
}
rc = syscall(SYS_arch_prctl, ARCH_GET_XCOMP_PERM, &bitmask);
if (rc) {
printf("rc error\n");
} else {
if (( bitmask & XFEATURE_MASK_XTILEDATA) == 0){
printf("verify AMX permission faild bitmask %ld\n",bitmask);
}
}
}
```
This request fails on the account of too small a size for sigaltstack used by CPython allocated in Modules/faulthandler.c
The stack size used is 2*SIGSTKSZ, and does not take hardware capabilities into account.
Linux kernel 5.14 adds support to query minimum size of sigaltstack dynamically via getauxval(AT_MINSIGSTKSZ).
AMX support is added in Linux kernel 5.16
CPython should make use of this when built against more recent Linux kernels.
|
msg414938 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-11 22:01 |
New changeset dc374ac7b0fabaed49461a2044c220765f48d229 by Victor Stinner in branch 'main':
bpo-46968: Add os.sysconf_names['SC_MINSIGSTKSZ'] (GH-31824)
https://github.com/python/cpython/commit/dc374ac7b0fabaed49461a2044c220765f48d229
|
msg414939 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-11 22:19 |
New changeset 3b128c054885fe881c3b57a5978de3ea89c81a9c by Oleksandr Pavlyk in branch 'main':
bpo-46968: Fix faulthandler for Sapphire Rapids Xeon (GH-31789)
https://github.com/python/cpython/commit/3b128c054885fe881c3b57a5978de3ea89c81a9c
|
msg414941 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-11 22:30 |
The latest faulthandler bug on sigaltstack() was bpo-21131 about FPU state stored by "XSAVE" and a Linux kernel change to fix a security vulnerability:
https://bugs.python.org/issue21131#msg349688
|
msg414943 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-11 23:04 |
New changeset 393e2bf6bc6effbfe821f051a230978f0edd70df by Victor Stinner in branch '3.10':
bpo-46968: Fix faulthandler for Sapphire Rapids Xeon (GH-31789) (GH-31830)
https://github.com/python/cpython/commit/393e2bf6bc6effbfe821f051a230978f0edd70df
|
msg414949 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-11 23:37 |
New changeset ba2b7956fa3932769a5c0aa2575de5c8d7e7ba4b by Victor Stinner in branch '3.9':
bpo-46968: Fix faulthandler for Sapphire Rapids Xeon (GH-31789) (GH-31831)
https://github.com/python/cpython/commit/ba2b7956fa3932769a5c0aa2575de5c8d7e7ba4b
|
msg414950 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-11 23:38 |
Thanks Oleksandr Pavlyk for your nice enhancement!
I merged your change to 3.9, 3.10 and main branches! I also added os.sysconf('SC_MINSIGSTKSZ') to Python 3.11.
|
msg414957 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-12 00:47 |
On my x86-64 Fedora 35 with the CPU "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz", I get:
* SIGSTKSZ = 8192
* getauxval(AT_MINSIGSTKSZ) = 2032
* faulthandler stack.ss_size = 10224
* os.sysconf('SC_MINSIGSTKSZ') = 2032
In C, sysconf(_SC_MINSIGSTKSZ) is similar to getauxval(AT_MINSIGSTKSZ), but the glibc sysconf(_SC_MINSIGSTKSZ) is more generic, whereas IMO getauxval(AT_MINSIGSTKSZ) is more the low-level Linux API.
|
msg414960 - (view) |
Author: Oleksandr Pavlyk (oleksandr-pavlyk) * |
Date: 2022-03-12 01:10 |
So where getauxval(AT_MINSIGSTKSZ) < SIGSTKSZ the merged changes actually resulted in decrease of the allocated signal deliver stack.
On Sapphire Rapids due to tile registers size we have getauxval(AT_MINSIGSTKSZ) > SIGSTKSZ.
This is why the initial proposal was to use SIGSTKSZ + max(getauxval(AT_MINSIGSTKSZ), SIGSTKSZ).
I suppose, ultimately I am saying that we should check that bpo-21131 does not regress.
|
msg414965 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-12 01:31 |
> So where getauxval(AT_MINSIGSTKSZ) < SIGSTKSZ the merged changes actually resulted in decrease of the allocated signal deliver stack.
faulthandler stack is only used in the least likely case: on a fatal error. It should reduce its memory footprint, so it's good that it uses less memory.
To fix bpo-21131, I chose to use SIGSTKSZ * 2 because:
* the fix is trivial: add "* 2"
* it "just works"
* there was no API to query how many bytes Linux uses on the stack
Previously, faulthandler had between 0 and SIGSTKSZ bytes depending on which CPU extension was used or not on the process.
If I understood correctly, on Linux 5.14 with the change, faulthandler now always has SIGSTKSZ bytes for its own usage, and the other bytes are used by the Linux kernel. So it's more reliable for faulthandler, to always have the same amount of memory for its personal use.
|
msg415402 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2022-03-17 12:48 |
Commit 393e2bf6bc6effbfe821f051a230978f0edd70df has broken CPython in RedHat 6:
[2022-03-16T18:49:20.608Z] 2022/03/16 14:48:55 ERROR /tmp/python3.10-3.10.3-0/Modules/faulthandler.c:28:12: fatal error: sys/auxv.h: No such file or directory
# include <sys/auxv.h>
^~~~~~~~~~~~
compilation terminated.
|
msg415403 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2022-03-17 12:49 |
This is problematic because this has been backported to stable releases.
|
msg415404 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2022-03-17 12:49 |
We may need to revert these commits and do another release.... sigh :(
|
msg415405 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2022-03-17 12:54 |
The configure file is checking for "linux/auxvec.h"
checking linux/auxvec.h usability... yes
but the code is including "sys/auxvec.h"
|
msg415406 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2022-03-17 12:55 |
The code is assuming that if linux/auxvec.h then sys/auxv.h will be available, which is wrong.
|
msg415407 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2022-03-17 13:02 |
This may be quite bad, because this means that 3.10 and 3.9 doesn't build in CentOS 6, which is used for manylinux2010 wheels
|
msg415483 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-18 10:33 |
> Commit 393e2bf6bc6effbfe821f051a230978f0edd70df has broken CPython in RedHat 6
Too bad that we want to support RHEL 6 but have no buildbot for that.
|
msg415487 - (view) |
Author: miss-islington (miss-islington) |
Date: 2022-03-18 12:03 |
New changeset 8e3fde728f547f1d32bde8adf62b4c50bb877b9d by Pablo Galindo Salgado in branch 'main':
bpo-46968: Check for 'sys/auxv.h' in the configure script (GH-31961)
https://github.com/python/cpython/commit/8e3fde728f547f1d32bde8adf62b4c50bb877b9d
|
msg415496 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2022-03-18 13:33 |
New changeset a12ef81231d65da5efbef4fa1434716270a19af6 by Pablo Galindo Salgado in branch '3.9':
[3.9] bpo-46968: Check for 'sys/auxv.h' in the configure script (GH-31961). (GH-31975)
https://github.com/python/cpython/commit/a12ef81231d65da5efbef4fa1434716270a19af6
|
msg415497 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2022-03-18 13:33 |
New changeset 6fd9737373f2bed03f409440b4fd50b9f8f121cb by Pablo Galindo Salgado in branch '3.10':
[3.10] bpo-46968: Check for 'sys/auxv.h' in the configure script (GH-31961). (GH-31974)
https://github.com/python/cpython/commit/6fd9737373f2bed03f409440b4fd50b9f8f121cb
|
msg415500 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2022-03-18 14:15 |
Thanks for the fix Pablo.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:59:57 | admin | set | github: 91124 |
2022-03-18 14:15:32 | vstinner | set | priority: release blocker ->
messages:
+ msg415500 |
2022-03-18 13:33:30 | pablogsal | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
2022-03-18 13:33:14 | pablogsal | set | messages:
+ msg415497 |
2022-03-18 13:33:11 | pablogsal | set | messages:
+ msg415496 |
2022-03-18 12:39:42 | pablogsal | set | pull_requests:
+ pull_request30065 |
2022-03-18 12:38:00 | pablogsal | set | pull_requests:
+ pull_request30064 |
2022-03-18 12:05:08 | lkollar | set | nosy:
+ lkollar
|
2022-03-18 12:03:28 | miss-islington | set | nosy:
+ miss-islington messages:
+ msg415487
|
2022-03-18 10:33:55 | vstinner | set | messages:
+ msg415483 |
2022-03-17 14:25:27 | pablogsal | set | stage: resolved -> patch review pull_requests:
+ pull_request30049 |
2022-03-17 13:02:03 | pablogsal | set | messages:
+ msg415407 |
2022-03-17 12:56:21 | pablogsal | set | priority: normal -> release blocker |
2022-03-17 12:55:19 | pablogsal | set | messages:
+ msg415406 |
2022-03-17 12:54:32 | pablogsal | set | messages:
+ msg415405 |
2022-03-17 12:49:59 | pablogsal | set | messages:
+ msg415404 |
2022-03-17 12:49:44 | pablogsal | set | nosy:
+ lukasz.langa messages:
+ msg415403
|
2022-03-17 12:48:15 | pablogsal | set | status: closed -> open
nosy:
+ pablogsal messages:
+ msg415402
resolution: fixed -> (no value) |
2022-03-12 01:31:53 | vstinner | set | messages:
+ msg414965 |
2022-03-12 01:10:23 | oleksandr-pavlyk | set | messages:
+ msg414960 |
2022-03-12 00:47:37 | vstinner | set | messages:
+ msg414957 |
2022-03-11 23:38:59 | vstinner | set | status: open -> closed versions:
- Python 3.7, Python 3.8 messages:
+ msg414950
resolution: fixed stage: patch review -> resolved |
2022-03-11 23:37:38 | vstinner | set | messages:
+ msg414949 |
2022-03-11 23:13:06 | vstinner | set | pull_requests:
+ pull_request29928 |
2022-03-11 23:04:29 | vstinner | set | messages:
+ msg414943 |
2022-03-11 22:30:19 | vstinner | set | messages:
+ msg414941 |
2022-03-11 22:22:48 | vstinner | set | pull_requests:
+ pull_request29927 |
2022-03-11 22:19:43 | vstinner | set | messages:
+ msg414939 |
2022-03-11 22:01:44 | vstinner | set | messages:
+ msg414938 |
2022-03-11 16:58:08 | vstinner | set | pull_requests:
+ pull_request29921 |
2022-03-10 06:32:44 | kumaraditya | set | nosy:
+ vstinner
|
2022-03-09 19:29:32 | oleksandr-pavlyk | set | keywords:
+ patch stage: patch review pull_requests:
+ pull_request29893 |
2022-03-09 19:07:02 | oleksandr-pavlyk | create | |