classification
Title: test_aead_aes_gcm fails on Kernel 4.9
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: barry, christian.heimes, cstratak, matejcik, nascheme, vstinner
Priority: normal Keywords: patch

Created on 2017-01-19 15:01 by christian.heimes, last changed 2017-10-09 15:28 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
test_socket_aead_kernel49.patch matejcik, 2017-02-02 17:37 updating AEAD test for kernel 4.9
Pull Requests
URL Status Linked Edit
PR 133 merged matejcik, 2017-02-16 13:26
Messages (14)
msg285794 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-01-19 15:01
$ ./python -m test -m test_aead_aes_gcm test_socket
Run tests sequentially
0:00:00 [1/1] test_socket
test test_socket failed -- Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/support/__init__.py", line 556, in wrapper
    return func(*args, **kw)
  File "/home/heimes/dev/python/cpython/Lib/test/test_socket.py", line 5515, in test_aead_aes_gcm
    res = op.recv(assoclen + len(plain) + taglen)
OSError: [Errno 22] Invalid argument

The tests were written and passed under Linux Kernel 4.7. I was under the assumption that the API is stable. But the most recent version 4.9 has changed the API for AEAD mode slightly. libkcapi project maintain a well written documentation of the Kernel crypto API,
https://github.com/smuellerDD/libkcapi/commit/be242c387b7030cbccae2c183107efa86d9a3cd6

Fedora 24 and 25 are affected. I'm going to update the tests. Downstream distributors: Feel free to disable the test in the mean time. The feature is not critical.
msg285797 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-01-19 15:39
recv(64) works. I need to figure out why 64 and what's in the extra bytes.
msg286808 - (view) Author: jan matejek (matejcik) * Date: 2017-02-02 16:55
The "'0' * taglen" part is now considered part of plaintext. Which makes a lot of sense :) 
Removing the "empty taglen" fixes the encryption part of the tests for me. Similarly, for the decryption test, we must only read and check the message without the tag.
msg286811 - (view) Author: jan matejek (matejcik) * Date: 2017-02-02 17:37
the attached patch fixes the test for me
msg287509 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-10 10:50
I confirm that test_socket pass with test_socket_aead_kernel49.patch on my Fedora 25 (kernel 4.9.5-200.fc25.x86_64).

I would be nice to fix test_socket which is currently broken ;-)
msg287614 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-02-11 17:20
Confirmed that test_socket_aead_kernel49.patch fixes the problem for Ubuntu 17.04.  It'll probably fix it for Debian Stretch too give its kernel version number, but I haven't tested that yet.
msg287616 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-02-11 18:08
Thx Barry,

I'll try to find some time to verify the new Kernel API and commit the patch early next week.
msg287902 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-15 23:41
@matejcik: would you mind to create a pull request for your patch?
msg287943 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-16 13:43
Even when the fix will be backported to Python 3.6, please keep the issue open until Christian Heimes has time to review the kernel changes. Copy/paste of his message on the PR#133:

"I still haven't had time to study the changes in the Linux kernel. Let's merge the PR to silence the test error. Please leave the ticket open as reminder for me."
msg295561 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2017-06-09 17:55
This patch works for me as well.  Debian testing, kernel:
Linux version 4.9.0-3-amd64 (debian-kernel@lists.debian.org) (gcc version 6.3.0 20170425 (Debian 6.3.0-16) ) #1 SMP Debian 4.9.25-1 (2017-05-02)


In the 3.6.1 Python release, test_socket fails with:

test test_socket failed -- Traceback (most recent call last):
  File "/home/nas/src/cpython/Lib/test/support/__init__.py", line 556, in wrapper
    return func(*args, **kw)
  File "/home/nas/src/cpython/Lib/test/test_socket.py", line 5497, in test_aead_aes_gcm
    res = op.recv(assoclen + len(plain) + taglen)
OSError: [Errno 22] Invalid argument
msg295569 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-06-09 20:40
The fix was also merged to master. It should be backported to 3.6. Neil, do
you want to try to backport it?
msg295583 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2017-06-09 21:30
This bug can be closed.  The fix is in the master branch (commit 9764c151c51480a) and has also been backported to the "3.6" branch (git commit 4ac01f0ff3e).  The fork for 3.6.1rc1 happened before the backport which is why 3.6.1 doesn't have the fix.
msg295621 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-06-10 08:53
Oh ok, fine. Hopefully, 3.6.2 is coming in next days ;-)
msg303976 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-10-09 15:28
FYI the merged fix is:

commit 9764c151c51480a7ca6042b1ccd69be2620ff360
Author: matejcik <matejcik@gmail.com>
Date:   Thu Feb 16 14:41:31 2017 +0100

    update test_socket AEAD test for kernel 4.9 and up (#133)
History
Date User Action Args
2017-10-09 15:28:12vstinnersetmessages: + msg303976
2017-06-10 08:53:18vstinnersetmessages: + msg295621
2017-06-09 21:30:06naschemesetstatus: open -> closed
resolution: fixed
messages: + msg295583

stage: resolved
2017-06-09 20:40:01vstinnersetmessages: + msg295569
2017-06-09 17:55:00naschemesetnosy: + nascheme
messages: + msg295561
2017-02-16 13:43:01vstinnersetmessages: + msg287943
2017-02-16 13:26:16matejciksetpull_requests: + pull_request94
2017-02-15 23:41:02vstinnersetmessages: + msg287902
2017-02-11 18:08:13christian.heimessetmessages: + msg287616
2017-02-11 17:20:56barrysetmessages: + msg287614
2017-02-11 17:14:11barrysetnosy: + barry
2017-02-10 10:50:09vstinnersetnosy: + vstinner
messages: + msg287509
2017-02-02 17:37:08matejciksetfiles: + test_socket_aead_kernel49.patch
keywords: + patch
messages: + msg286811
2017-02-02 16:55:43matejciksetnosy: + matejcik
messages: + msg286808
2017-01-19 15:39:01christian.heimessetmessages: + msg285797
2017-01-19 15:03:17cstrataksetnosy: + cstratak
2017-01-19 15:01:06christian.heimescreate