msg272516 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2016-08-12 10:21 |
Linux has a netlink-based user-space interface for Kernel cryptography. Kernel based crypto has a couple of advantages that are explained at http://www.chronox.de/libkcapi/html/ch01s02.html . The document doesn't mention that a crypto socket also supports splicing and sendfile. Files no longer have to be copied to user-space.
My experimental branch https://github.com/tiran/cpython/commits/feature/af_alg implements af_alg support. Example:
from socket import socket, AF_ALG, SOCK_SEQPACKET, SOL_ALG, ALG_SET_KEY
from binascii import hexlify
with socket(AF_ALG, SOCK_SEQPACKET, 0) as alg:
alg.bind(('hash', 'hmac(sha512)'))
alg.setsockopt(SOL_ALG, ALG_SET_KEY, b'key')
op, _ = alg.accept()
with open('/etc/passwd', 'rb') as f:
op.sendfile(f)
print(hexlify(op.recv(64)))
op.close()
|
msg272746 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2016-08-15 10:30 |
Working patch with tests and documentation.
socket.algset() isn't strictly necessary but makes the feature much more pleasant to use. I accept ideas for a better name, though.
|
msg272918 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2016-08-17 11:20 |
I reviewed AF_ALG-kernel-crypto-support-for-socket-module.patch.
|
msg273018 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2016-08-18 10:49 |
Thanks for your review, Victor. I have addressed most of your remarks.
* algset() is now called sendmsg_afalg(). It behaves more like a specialized version of sendmsg() and can optionally handle an array of iovec.
* I had to add another variant of setsockopt that sends NULL, int instead of (char*)int, sizeof(int) to get the AEAD GCM tests working. AEAD expects ALG_SET_AEAD_AUTHSIZE as (NULL, taglen). algo.setsockopt(SOL_ALG, ALG_SET_AEAD_AUTHSIZE, (None, taglen)) sends optval=NULL, optlen=taglen.
* Added tests for AES-CBC decryption, AEAD AES-GCM and DRBG.
|
msg273838 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2016-08-28 19:37 |
New patch with setsockopt(socket.SOL_ALG, socket.ALG_SET_AEAD_AUTHSIZE, None, taglen) instead of (None, taglen).
|
msg274365 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2016-09-04 15:13 |
I have removed binascii.(un)hexlify().
|
msg274442 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2016-09-05 21:55 |
New changeset 74ce062a0397 by Christian Heimes in branch 'default':
Issue #27744: Add AF_ALG (Linux Kernel crypto) to socket module.
https://hg.python.org/cpython/rev/74ce062a0397
|
msg274445 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2016-09-05 22:07 |
New changeset 52404f9596b5 by Christian Heimes in branch 'default':
Issue #27744: correct comment and markup
https://hg.python.org/cpython/rev/52404f9596b5
|
msg274453 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2016-09-05 22:38 |
New changeset ee32af890e27 by Christian Heimes in branch 'default':
Issue 27744: Check for AF_ALG support in Kernel
https://hg.python.org/cpython/rev/ee32af890e27
|
msg274458 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2016-09-05 22:59 |
New changeset 4ebe3ade6922 by Christian Heimes in branch 'default':
Issue 27744: AES-CBC and DRBG need Kernel 3.19+
https://hg.python.org/cpython/rev/4ebe3ade6922
|
msg274508 - (view) |
Author: Martin Panter (martin.panter) *  |
Date: 2016-09-06 03:33 |
Despite the last changes, test_aes_cbc() hangs for fifteen minutes:
http://buildbot.python.org/all/builders/x86-64%20Ubuntu%2015.10%20Skylake%20CPU%203.x/builds/1298/steps/test/logs/stdio
running: test_socket (900 sec)
0:28:47 [332/402] test_socket crashed
Timeout (0:15:00)!
Thread 0x00007f2843d94700 (most recent call first):
File "/home/buildbot/buildarea/3.x.intel-ubuntu-skylake/build/Lib/test/test_socket.py", line 5389 in test_aes_cbc
File "/home/buildbot/buildarea/3.x.intel-ubuntu-skylake/build/Lib/test/support/__init__.py", line 523 in wrapper
File "/home/buildbot/buildarea/3.x.intel-ubuntu-skylake/build/Lib/unittest/case.py", line 600 in run
. . .
|
msg274516 - (view) |
Author: Martin Panter (martin.panter) *  |
Date: 2016-09-06 04:37 |
Also, the Gentoo buildbots fail:
http://buildbot.python.org/all/builders/x86%20Gentoo%20Non-Debug%20with%20X%203.x/builds/1368/steps/test/logs/stdio
======================================================================
ERROR: test_aead_aes_gcm (test.test_socket.LinuxKernelCryptoAPI)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/support/__init__.py", line 523, in wrapper
return func(*args, **kw)
File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_socket.py", line 5428, in test_aead_aes_gcm
with self.create_alg('aead', 'gcm(aes)') as algo:
File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_socket.py", line 5346, in create_alg
sock.bind((typ, name))
FileNotFoundError: [Errno 2] No such file or directory
Similar failures for test_aes_cbc test_drbg_pr_sha256 test_hmac_sha1 test_sha256.
|
msg274520 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2016-09-06 04:53 |
> Also, the Gentoo buildbots fail:
> http://buildbot.python.org/all/builders/x86%20Gentoo%20Non-Debug%20with%20X%203.x/builds/1368/steps/test/logs/stdio
> ======================================================================
> ERROR: test_aead_aes_gcm (test.test_socket.LinuxKernelCryptoAPI)
It's Linux 4.4.6. configure says "checking for sockaddr_alg... yes".
|
msg274543 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2016-09-06 09:14 |
New changeset e3b83bfa02c5 by Christian Heimes in branch 'default':
Issue 27744: skip test if AF_ALG socket bind fails
https://hg.python.org/cpython/rev/e3b83bfa02c5
|
msg274553 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2016-09-06 11:31 |
Some distributions mess with the Kernel or disable user-space crypto. I have added some tweaks and fixed a couple of buildbots. I don't know what is going on with x86-64 Ubuntu 15.10 Skylake CPU. It's a Kernel 4.2 machine and should support AES-CBC.
|
msg274557 - (view) |
Author: Xiang Zhang (xiang.zhang) *  |
Date: 2016-09-06 14:20 |
My PC is Ubuntu15.10, kernel 4.2, though CPU not Skylake. Everything works fine.
test_aead_aes_gcm (test.test_socket.LinuxKernelCryptoAPI) ... skipped "('[Errno 2] No such file or directory', 'aead', 'gcm(aes)')"
test_aes_cbc (test.test_socket.LinuxKernelCryptoAPI) ... ok
test_drbg_pr_sha256 (test.test_socket.LinuxKernelCryptoAPI) ... ok
test_hmac_sha1 (test.test_socket.LinuxKernelCryptoAPI) ... ok
test_sendmsg_afalg_args (test.test_socket.LinuxKernelCryptoAPI) ... ok
test_sha256 (test.test_socket.LinuxKernelCryptoAPI) ... ok
|
msg275702 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2016-09-10 21:32 |
x86-64 Ubuntu 15.10 Skylake CPU 3.x is still blocking. It looks like I have to add another workaround for a Ubuntu quirk.
|
msg275824 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2016-09-11 18:11 |
New changeset 55d77f5a7cb3 by Christian Heimes in branch 'default':
Issue 27744: test_aes_cbc is blocking x86-64 Ubuntu 15.10 Skylake CPU 3.x for a while. Require Kernel 4.3+ for now
https://hg.python.org/cpython/rev/55d77f5a7cb3
|
msg276010 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2016-09-12 09:50 |
New changeset a951f8f30922 by Victor Stinner in branch 'default':
Cleanup socketmodule.c
https://hg.python.org/cpython/rev/a951f8f30922
New changeset 3a6917c73857 by Victor Stinner in branch 'default':
socket: Fix memory leak in sendmsg() and sendmsg_afalg()
https://hg.python.org/cpython/rev/3a6917c73857
|
msg287497 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2017-02-10 09:39 |
What is the status of this issue? test_aead_aes_gcm() fails on my Fedora 25 (Python: default branch).
haypo@selma$ cat /etc/fedora-release
Fedora release 25 (Twenty Five)
haypo@selma$ uname -r
4.9.5-200.fc25.x86_64
test test_socket failed -- Traceback (most recent call last):
File "/home/haypo/prog/python/default/Lib/test/support/__init__.py", line 556, in wrapper
return func(*args, **kw)
File "/home/haypo/prog/python/default/Lib/test/test_socket.py", line 5515, in test_aead_aes_gcm
res = op.recv(assoclen + len(plain) + taglen)
OSError: [Errno 22] Invalid argument
|
msg287502 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2017-02-10 10:06 |
I'll look into the matter and push a fix after the migration to github today.
|
msg287503 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2017-02-10 10:08 |
By the way problem with AES-GCM is tracked in https://bugs.python.org/issue29324 . It was caused in a Kernel API change. Jan has provided a fix. I need to find some spare time to dig into Kernel sources and verify the patch.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:58:34 | admin | set | github: 71931 |
2021-10-12 09:36:25 | christian.heimes | set | status: open -> closed resolution: fixed stage: resolved |
2017-02-10 10:08:25 | christian.heimes | set | messages:
+ msg287503 |
2017-02-10 10:06:41 | christian.heimes | set | messages:
+ msg287502 |
2017-02-10 09:39:45 | vstinner | set | messages:
+ msg287497 |
2017-02-03 06:55:56 | Christian H | set | nosy:
+ Christian H
|
2016-09-12 09:50:31 | python-dev | set | messages:
+ msg276010 |
2016-09-11 18:11:48 | python-dev | set | messages:
+ msg275824 |
2016-09-10 21:32:58 | christian.heimes | set | messages:
+ msg275702 |
2016-09-06 14:20:18 | xiang.zhang | set | nosy:
+ xiang.zhang messages:
+ msg274557
|
2016-09-06 11:31:22 | christian.heimes | set | messages:
+ msg274553 |
2016-09-06 09:14:23 | python-dev | set | messages:
+ msg274543 |
2016-09-06 04:53:48 | vstinner | set | messages:
+ msg274520 |
2016-09-06 04:37:53 | martin.panter | set | messages:
+ msg274516 |
2016-09-06 03:33:19 | martin.panter | set | nosy:
+ martin.panter messages:
+ msg274508
|
2016-09-05 22:59:06 | python-dev | set | messages:
+ msg274458 |
2016-09-05 22:38:44 | python-dev | set | messages:
+ msg274453 |
2016-09-05 22:07:09 | python-dev | set | messages:
+ msg274445 |
2016-09-05 21:55:07 | python-dev | set | nosy:
+ python-dev messages:
+ msg274442
|
2016-09-04 15:13:55 | christian.heimes | set | files:
+ AF_ALG-kernel-crypto-support-for-socket-module-5.patch
messages:
+ msg274365 |
2016-08-31 14:03:18 | christian.heimes | set | files:
+ AF_ALG-kernel-crypto-support-for-socket-module-4.patch |
2016-08-30 13:50:24 | christian.heimes | set | files:
+ AF_ALG-kernel-crypto-support-for-socket-module-3.patch |
2016-08-28 19:37:42 | christian.heimes | set | files:
+ AF_ALG-kernel-crypto-support-for-socket-module-2.patch
messages:
+ msg273838 |
2016-08-18 10:49:30 | christian.heimes | set | files:
+ AF_ALG-kernel-crypto-support-for-socket-module-1.patch
messages:
+ msg273018 |
2016-08-17 11:20:39 | vstinner | set | nosy:
+ vstinner messages:
+ msg272918
|
2016-08-15 10:30:44 | christian.heimes | set | files:
+ AF_ALG-kernel-crypto-support-for-socket-module.patch keywords:
+ patch messages:
+ msg272746
|
2016-08-12 13:41:46 | christian.heimes | set | files:
- Microsoft_Screen_Sharing_for_Lumia_Phones_HD-10_UG_th_TH.pdf |
2016-08-12 13:40:51 | Pan Naekton | set | files:
+ Microsoft_Screen_Sharing_for_Lumia_Phones_HD-10_UG_th_TH.pdf |
2016-08-12 10:37:39 | Lukasa | set | nosy:
+ Lukasa
|
2016-08-12 10:21:30 | christian.heimes | create | |