classification
Title: test_ssl test_options fails on ubuntu 16.04
Type: behavior Stage: resolved
Components: Library (Lib), SSL Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: SilentGhost, alex, benjamin.peterson, christian.heimes, doko, dstufft, giampaolo.rodola, janssen, larry, martin.panter, ned.deily, pitrou, python-dev, xiang.zhang
Priority: normal Keywords:

Created on 2016-04-27 03:34 by xiang.zhang, last changed 2017-03-24 23:31 by xiang.zhang. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 374 merged xiang.zhang, 2017-03-01 06:42
Messages (20)
msg264342 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-04-27 03:37
test_options in test_ssl fails on Ubuntu 16.04. I don't know this is due to the newest ubuntu or a recent code change. But I checkout revision 90000 and then rebuild and test, test_option still fails.

The traceback is:

FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/angwer/cpython/Lib/test/test_ssl.py", line 847, in test_options
    self.assertEqual(0, ctx.options)
AssertionError: 0 != 33554432
msg264414 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-04-28 05:13
After some test, I think the reason causing this error is due to SSL_CTX_clear_options. 

With OPENSSL_VERSION_NUMBER 268443775, SSL_CTX_clear_options(self->ctx, 2248147967) returns 33554432, where SSL_CTX_get_options returns 2248147967. From the manpage of SSL_CTX_clear_options, it seems it should return 0.
msg264417 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-04-28 05:59
From the source code (get from apt-get source) of openssl-1.0.2g, I find

SSL_CTX_clear_options(ctx, op):
  op &= ~SSL_OP_NO_SSLv3
  return (ctx->options &= ~op)
SSL_CTX_set_options(ctx, op):
  op |= SSL_OP_NO_SSLv3
  return (ctx->options |= op)

which differs from the official code repos:

SSL_CTX_clear_options(ctx, op):
  return (ctx->options &= ~op)
SSL_CTX_set_options(ctx, op):
  return (ctx->options |= op)

This difference is introduced by debian-specific patch:

     case SSL_CTRL_OPTIONS:
+        larg|=SSL_OP_NO_SSLv3;
         return (ctx->options |= larg);
     case SSL_CTRL_CLEAR_OPTIONS:
+        larg&=~SSL_OP_NO_SSLv3;
         return (ctx->options &= ~larg);
msg267472 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-06-05 20:31
Can we close this as an Ubuntu-specific problem?
msg267485 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-06 00:02
This test is already decorated with @skip_if_broken_ubuntu_ssl. I’m not sure Python should go too far out of its way to handle downstream patches, but it seems there is a precedent here.
msg267488 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-06-06 00:07
I just spoke with @doko about this here at PyCon.  I think we came to the conclusion it might be time to consider removing the old @skip_if_broken_ubuntu_ssl decorator and focus on making the tests work with the most recent releases since pretty much every distributor and current Python releases have moved to disabling the old compromised ssl/tls versions.
msg267503 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-06-06 02:14
@skip_if_broken_ubuntu_ssl doesn't work in this case. `hasattr(ssl, 'PROTOCOL_SSLv2')` returns False.
msg268322 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-12 04:41
I got this when testing 3.5.2rc1 on my Ubuntu 16.04 machine.  CAs Xiang Zhang showed, this is Ubuntu doing something crazy.  I ignored the failure and shipped 3.5.2rc1, however I would be interested in suppressing the test for 3.5.2 final.  That way it has a chance of passing the whole test suite on user's Linux machines...!
msg268325 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-06-12 04:59
ubuntu doesn't do anything crazy, but just disables oldish, deprecated und probably now unsecure ssl protocols.  This is done by other vendors as well.   From my point of of view this skip_if_ubuntu stuff should be replaced by proper feature tests.  I'll see if I can come up with another work around.
msg268327 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-12 05:30
This still affects 3.4 and 3.5.  It'd be lovely if it could be fixed in all the still-alive versions.  (Yes, this is technically a "bug fix", but I'd still like it fixed in 3.4.)
msg268425 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-06-13 06:40
Description: properly handle Ubuntu's openssl having OP_NO_SSLv3 forced on by default
Author: Marc Deslauriers <marc.deslauriers@canonical.com>
Forwarded: yes, http://bugs.python.org/issue25724

Index: b/Lib/test/test_ssl.py
===================================================================
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -821,7 +821,8 @@ class ContextTests(unittest.TestCase):
             self.assertEqual(ssl.OP_ALL | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv3,
                              ctx.options)
             ctx.options = 0
-            self.assertEqual(0, ctx.options)
+            # Ubuntu has OP_NO_SSLv3 forced on by default
+            self.assertEqual(0, ctx.options & ~ssl.OP_NO_SSLv3)
         else:
             with self.assertRaises(ValueError):
                 ctx.options = 0
msg268426 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-13 06:47
That does seem like it'd make the test failure go away.  But the fix seems a little Ubuntu-specific.  Is it reasonable to do that when testing on every platform?
msg268706 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-17 04:23
FWIW I imagine Ubuntu overriding the option will break the example code in the documentation of clearing SSL_OP_NO_SSLv3: <https://docs.python.org/3.5/library/ssl.html#ssl.create_default_context>. If we keep that documentation, I think we should continue to test that clearing the option works, which conflicts with the proposed patch.
msg269165 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-24 07:04
Well, I want this fixed in 3.5.2 final.  If nobody can propose a better patch in the next 24 hours then I'm going with Matthias's patch.
msg269216 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-24 23:46
FWIW I had a quick look at ways to detect if you are running on Ubuntu. But platform.linux_distribution() seems to be deprecated and looks like it might have trouble differentiating Debian and Ubuntu. So it may be easier to just go with the current patch on all platforms, at least for the moment.

Maybe if someone that uses Ubuntu could suggest a specific file or config the test can check for.
msg269261 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-25 21:16
Well, as Donald Rumsfeld said in 2008: "As you know, you go to war with the army you have, not the army you might want or wish to have at a later time."

3.5.2 final and 3.4.5 final will ship with Matthias's patch as proposed.  FWIW I'd accept an improved patch in both versions for the next release.
msg269341 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-27 03:02
New changeset 4d04aca4afb0 by Matthias Klose in branch '3.5':
Issue #26867: Ubuntu's openssl OP_NO_SSLv3 is forced on by default; fix test.
https://hg.python.org/cpython/rev/4d04aca4afb0

New changeset 8f028d04df11 by Matthias Klose in branch '3.4':
Issue #26867: Ubuntu's openssl OP_NO_SSLv3 is forced on by default; fix test.
https://hg.python.org/cpython/rev/8f028d04df11
msg270154 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-07-11 02:47
Does this need to be backport to py2.7? It suffers from the same problem.
msg284713 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-01-05 07:43
This test still fails with lastest Py2.7 on Ubuntu 16.10. Could we backport the patch to silence the failure?

./python -m test.regrtest test_ssl
[1/1] test_ssl
test test_ssl failed -- Traceback (most recent call last):
  File "/home/angwer/py2.7/Lib/test/test_ssl.py", line 780, in test_options
    self.assertEqual(0, ctx.options)
AssertionError: 0 != 33554432L

1 test failed:
    test_ssl
msg290355 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-03-24 23:31
New changeset c9ba1862222bcbb309278db028d33a57f039d587 by Xiang Zhang in branch '2.7':
bpo-26867: Ubuntu's openssl OP_NO_SSLv3 is forced on by default; fix test. (GH-374)
https://github.com/python/cpython/commit/c9ba1862222bcbb309278db028d33a57f039d587
History
Date User Action Args
2017-03-24 23:31:27xiang.zhangsetmessages: + msg290355
2017-03-01 07:36:50xiang.zhangsetstatus: open -> closed
stage: resolved
resolution: fixed
versions: + Python 2.7
2017-03-01 06:42:11xiang.zhangsetpull_requests: + pull_request311
2017-01-05 07:43:16xiang.zhangsetnosy: + benjamin.peterson
messages: + msg284713
2016-09-15 08:02:36christian.heimessetassignee: christian.heimes

components: + SSL
nosy: + christian.heimes
2016-07-11 02:47:00xiang.zhangsetmessages: + msg270154
2016-06-27 03:02:12python-devsetnosy: + python-dev
messages: + msg269341
2016-06-25 21:16:39larrysetmessages: + msg269261
2016-06-24 23:46:32martin.pantersetmessages: + msg269216
2016-06-24 07:04:53larrysetmessages: + msg269165
2016-06-17 04:23:13martin.pantersetmessages: + msg268706
2016-06-17 04:15:01martin.panterlinkissue25724 superseder
2016-06-13 06:47:04larrysetmessages: + msg268426
2016-06-13 06:40:13dokosetmessages: + msg268425
2016-06-12 11:33:28christian.heimessetnosy: - christian.heimes
2016-06-12 05:30:28larrysetmessages: + msg268327
versions: + Python 3.4, Python 3.5
2016-06-12 04:59:10dokosetmessages: + msg268325
2016-06-12 04:41:02larrysetnosy: + larry
messages: + msg268322
2016-06-06 07:48:54SilentGhostsetnosy: + SilentGhost

versions: + Python 3.6
2016-06-06 02:14:04xiang.zhangsetmessages: + msg267503
2016-06-06 00:07:16ned.deilysetmessages: + msg267488
2016-06-06 00:02:38martin.pantersetnosy: + martin.panter
messages: + msg267485
2016-06-05 20:31:39ned.deilysetnosy: + ned.deily, doko
messages: + msg267472
2016-04-28 05:59:26xiang.zhangsetmessages: + msg264417
2016-04-28 05:13:46xiang.zhangsetmessages: + msg264414
2016-04-27 03:37:18xiang.zhangsetnosy: + janssen, pitrou, giampaolo.rodola, christian.heimes, alex, dstufft
messages: + msg264342

components: + Library (Lib)
type: behavior
2016-04-27 03:34:14xiang.zhangcreate