classification
Title: Unable to copy ssl.SSLContext
Type: behavior Stage: patch review
Components: SSL Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, ned.deily, pitrou, serhiy.storchaka, vitaly.krug
Priority: normal Keywords: patch

Created on 2018-03-08 06:09 by vitaly.krug, last changed 2018-06-07 07:06 by serhiy.storchaka.

Pull Requests
URL Status Linked Edit
PR 6099 open christian.heimes, 2018-03-13 08:50
Messages (18)
msg313422 - (view) Author: Vitaly Kruglikov (vitaly.krug) Date: 2018-03-08 06:09
```
import copy
import ssl

copy.copy(ssl.create_default_context())
```
results in 

`TypeError: can't pickle SSLContext objects`

This prevents me from being able to `copy.deepcopy()` an object that references `ssl.SSLContext`.

The apparent root cause is apparently that `ssl.SSLContext` passes an extra arg to its `__new__` method, but doesn't implement the method `__getnewargs__` that would let `copy` extract the extra arg.
msg313437 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-03-08 12:30
This is rather a feature than a bug. It is not possible to make a copy of a SSLContext object because OpenSSL doesn't support the operation. A context contains elements that can't be cloned easily, e.g. session resumption tickets.
msg313458 - (view) Author: Vitaly Kruglikov (vitaly.krug) Date: 2018-03-08 21:37
Hi Christian, thank you for following up. Here is my use case, and perhaps you can suggest something else that will work:

I am refactoring the transport layer in the Pika AMQP client library. The user provides an ssl.SSLContext instance for connecting to an AMQP broker (e.g., RabbitMQ). Pika will resolve the hostname via getaddrinfo and make attempts to establish TCP and AMQP connection to the candidate IP addresses until one succeeds in establishing an AMQP connection over SSL. Each connection attempt will require a fresh unadulterated clone of the ssl.SSLContext instance provided by user to avoid any side-effects from prior connection attempts.

How can I obtain this pristine clone cleanly for each new connection attempt?
msg313459 - (view) Author: Vitaly Kruglikov (vitaly.krug) Date: 2018-03-08 21:40
Also, updating ssl.SSLContext documentation about intentional inability to copy generically and suggestion how to go about it if you need to obtain a clone or similar would be terrific and save developers time so they won't run into this gotcha when designing and implementing solutions. Many thanks!
msg313621 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-11 21:23
> Each connection attempt will require a fresh unadulterated clone of the ssl.SSLContext instance provided by user to avoid any side-effects from prior connection attempts.

What would those side-effects be?
msg313703 - (view) Author: Vitaly Kruglikov (vitaly.krug) Date: 2018-03-12 21:26
> What would those side-effects be?

Christian Heimes suggested that 
> A context contains elements that can't be cloned easily, e.g. session resumption tickets.

My concern then would be potential side-effects from such session resumption tickets and anything else that one connection attempt might save/change within an SSL Context that might have an undesirable side-effect on the follow-on connection attempts.
msg313704 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-12 21:30
You won't have any session resumption tickets until a connection succeeds.  And even then, I don't think it would be a problem.  By design, SSL contexts are meant to be re-used accross multiple connections.
msg313731 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-03-13 08:54
Antoine is correct. Session resumption is fully transparent for the application layer. The context object is designed to be reused by multiple connections to same or different servers.
msg314322 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-03-23 20:25
PR 6099 changes error message "can't pickle SSLContext objects" to "cannot serialize SSLContext object", right? Wouldn't be better to change the standard error message instead?
msg314347 - (view) Author: Vitaly Kruglikov (vitaly.krug) Date: 2018-03-24 00:35
It would be very helpful to make a statement in SSLContext's documentation to the effect that it's not copyable. This is frankly the first time I run into a non-copyable object.I spend quite a bit of time researching this after implementing a copying strategy that failed. It would have saved me (and others...) so much time is there was a warning in SSLContext documentation about not being able to serialize/copy/deepcopy by design!

Also, making that exception message more generic (ha, I wasn't pickling anything?!) as Serhiy Storchaka suggested would be a welcome addition, but not replacement for documentation.
msg314367 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-03-24 14:28
Serhiy,
I don't understand what you are trying to tell me. "cannot serialize '%s' object" is used all over the interpreter, e.g. io, pickle, etree, and more. I feel it's the standard message.

Vitaly,
A lot of objects can't be copied. It's the general case for all kinds of objects that hold operating system resources (files, sockets) or wrap external C libraries (bz2, lzma, sqlite, ssl). We generally don't document that an object cannot be pickled, serialized, or copied. If documentation doesn't state that an object is copy-able or doesn't provide a dedicated copy method, than it can't be copied.
msg314375 - (view) Author: Vitaly Kruglikov (vitaly.krug) Date: 2018-03-24 17:37
Thank you, I'll consider myself having been warned :)

On Sat, Mar 24, 2018, 7:28 AM Christian Heimes <report@bugs.python.org>
wrote:

>
> Christian Heimes <lists@cheimes.de> added the comment:
>
> Serhiy,
> I don't understand what you are trying to tell me. "cannot serialize '%s'
> object" is used all over the interpreter, e.g. io, pickle, etree, and more.
> I feel it's the standard message.
>
> Vitaly,
> A lot of objects can't be copied. It's the general case for all kinds of
> objects that hold operating system resources (files, sockets) or wrap
> external C libraries (bz2, lzma, sqlite, ssl). We generally don't document
> that an object cannot be pickled, serialized, or copied. If documentation
> doesn't state that an object is copy-able or doesn't provide a dedicated
> copy method, than it can't be copied.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue33023>
> _______________________________________
>
msg314382 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-03-24 19:24
Christian, I thought the reason of adding __getstate__ methods which raise an exception is that the existing error message doesn't satisfy your. What is the other reason?
msg314417 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-03-25 14:27
That's the only reason for PR 6099. The change makes it more obvious that SSL objects can't be serialized or copied.
msg314419 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-03-25 16:29
See issue33138.
msg318894 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-06-07 05:40
We have a languishing PR here.  What should be done with it?  If I understand correctly, Serhiy is proposing a more general cleanup in Issue33138 PR 6239.  If that is merged, can PR 6099 here be simplified?  What branches should it apply to? (retargeting for 3.8 pending comments)
msg318899 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-06-07 06:44
The change to the dup() method looks as a bug fix to me. It needs to be backported to all maintained versions. New tests can be backported to all versions except perhaps 2.7 (not tested). Changes to __getstate__() will be not needed if merge issue33138 PR 6239 (they just change the error message). But it may be needed to apply them to 2.7 only.
msg318903 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-06-07 07:06
If changes to __getstate__() be merged in 2.7, I suggest to unify them with issue33138.
History
Date User Action Args
2018-06-07 07:06:18serhiy.storchakasetmessages: + msg318903
2018-06-07 06:44:46serhiy.storchakasetmessages: + msg318899
2018-06-07 05:40:25ned.deilysetnosy: + ned.deily

messages: + msg318894
versions: + Python 3.8, - Python 2.7, Python 3.6
2018-03-25 16:29:21serhiy.storchakasetmessages: + msg314419
2018-03-25 14:27:06christian.heimessetmessages: + msg314417
2018-03-24 19:24:08serhiy.storchakasetmessages: + msg314382
2018-03-24 17:37:41vitaly.krugsetmessages: + msg314375
2018-03-24 14:28:33christian.heimessetmessages: + msg314367
2018-03-24 00:35:22vitaly.krugsetmessages: + msg314347
2018-03-23 20:25:41serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg314322
2018-03-13 08:54:03christian.heimessetmessages: + msg313731
2018-03-13 08:50:12christian.heimessetkeywords: + patch
stage: patch review
pull_requests: + pull_request5862
2018-03-12 21:30:14pitrousetmessages: + msg313704
2018-03-12 21:26:48vitaly.krugsetmessages: + msg313703
2018-03-11 21:24:04pitrousettype: crash -> behavior
2018-03-11 21:23:51pitrousetnosy: + pitrou
messages: + msg313621
2018-03-08 21:40:39vitaly.krugsetmessages: + msg313459
2018-03-08 21:37:09vitaly.krugsetmessages: + msg313458
2018-03-08 12:30:30christian.heimessetmessages: + msg313437
2018-03-08 06:10:00vitaly.krugsetassignee: christian.heimes

components: + SSL
nosy: + christian.heimes
2018-03-08 06:09:44vitaly.krugsettype: crash
2018-03-08 06:09:21vitaly.krugcreate