classification
Title: asyncio loop.start_tls() provide support for TLS in TLS
Type: Stage: patch review
Components: asyncio Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, cooperlees, fantix, jborean93, yselivanov
Priority: normal Keywords: patch

Created on 2019-06-06 17:55 by cooperlees, last changed 2021-08-30 20:18 by jborean93.

Pull Requests
URL Status Linked Edit
PR 17975 cooperlees, 2020-07-08 23:08
PR 26454 closed jborean93, 2021-05-30 02:32
PR 28073 open jborean93, 2021-08-30 20:18
Messages (10)
msg344846 - (view) Author: Cooper Lees (cooperlees) * Date: 2019-06-06 17:55
aiohttp would love to be able to support HTTPS Proxy servers. To do this, asyncio itself needs to be able to provide TLS within TLS connections.

Can we add this support to asyncio please.

(I tried search but could not find a related issue - Please merge if there is)
msg344849 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-06-06 18:29
Yeah, we have the SSL reimplementation ready to be backported from uvloop to cpython.  Fantix, the original author, should be able to do that soon.
msg346650 - (view) Author: Cooper Lees (cooperlees) * Date: 2019-06-26 16:29
@fantix - Is there anything I can do to help this progress. I'd be happy to potentially even do parts of the back porting if you're swamped. Would just need some guidance.

I need this as I'm looking to add auth to some internal HTTP(S) proxies I use and in order to allow aiohttp to be able to do client TLS auth, this is required.
msg373355 - (view) Author: Cooper Lees (cooperlees) * Date: 2020-07-08 21:29
Another bump since I've waiting over a year. Any plans for this? Will it make 3.10? Anything I can do?
msg373359 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2020-07-08 23:05
Looks like https://github.com/python/cpython/pull/17975 was forgotten and was never committed to 3.9. So it's 3.10 now.

Best bet for you is to use uvloop which should support the feature.
msg373364 - (view) Author: Cooper Lees (cooperlees) * Date: 2020-07-09 00:19
Yury, only problem with that is aiohttp hard blocks HTTPS proxies period. The aiohttp issue says they won't fix this until asyncio supports it. Kinda understand that.

[cooper:~]$ ./aioclient.par
HTTPS proxies https://fwdproxy:8082 are not supported, ignoring
^CTraceback (most recent call last):
  File "<string>", line 37, in <module>
  File "<string>", line 35, in __run
  File "/usr/local/fbcode/platform007/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/fbcode/platform007/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/data/users/cooper/fbsource/fbcode/buck-out/dev/gen/ti/fwdproxy/client_samples/py/aioclient#link-tree/ti/fwdproxy/client_samples/py/aioclient.py", line 56, in <module>
    asyncio.run(run_example())
  File "/usr/local/fbcode/platform007/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "uvloop/loop.pyx", line 1450, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1443, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1351, in uvloop.loop.Loop.run_forever
  File "uvloop/loop.pyx", line 519, in uvloop.loop.Loop._run
  File "uvloop/handles/poll.pyx", line 213, in uvloop.loop.__on_uvpoll_event
  File "uvloop/cbhandles.pyx", line 90, in uvloop.loop.Handle._run
  File "uvloop/cbhandles.pyx", line 73, in uvloop.loop.Handle._run
  File "uvloop/loop.pyx", line 359, in uvloop.loop.Loop._read_from_self
  File "uvloop/loop.pyx", line 364, in uvloop.loop.Loop._invoke_signals
  File "uvloop/loop.pyx", line 339, in uvloop.loop.Loop._ceval_process_signals
KeyboardInterrupt

Kept stack trace to prove I was using uvloop :)
msg373410 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2020-07-09 17:45
> The aiohttp issue says they won't fix this until asyncio supports it. Kinda understand that.

I saw you opened an issue with aiohttp to allow this and they're open to it. I hope that will get some movement. It also would be a big test for uvloop's (and 3.10 asyncio) TLS implementation.
msg390922 - (view) Author: Jordan Borean (jborean93) * Date: 2021-04-13 05:11
I'm looking through the PR https://github.com/python/cpython/pull/17975 and it doesn't look like it addresses this particular problem. The code for start_tls https://github.com/python/cpython/blob/d2a8e69c2c605fbaa3656a5f99aa8d295f74c80e/Lib/asyncio/base_events.py#L1210-L1212 has a check for the attribute '_start_tls_compatible' but _SSLProtocolTransport (https://github.com/python/cpython/blob/master/Lib/asyncio/sslproto.py) does not set this. The PR mentioned https://github.com/python/cpython/pull/17975 does not seem to touch any of this so I would assume the problem is still there?

I also installed 3.10.0a7 and this problem still persists when I call loop.start_tls with my SSL writer transport.
msg390934 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2021-04-13 07:59
Thanks for the reminder.
You are correct, the mentioned PR should set _SSLProtocolTransport._start_tls_compatible to True
msg390990 - (view) Author: Jordan Borean (jborean93) * Date: 2021-04-13 19:15
Fantastic thanks, I'll keep watching the issue in the background as it sounds like it's under control.
History
Date User Action Args
2021-08-30 20:18:41jborean93setpull_requests: + pull_request26517
2021-05-30 02:32:32jborean93setpull_requests: + pull_request25050
2021-04-13 19:15:40jborean93setmessages: + msg390990
2021-04-13 07:59:11asvetlovsetmessages: + msg390934
2021-04-13 05:11:38jborean93setnosy: + jborean93
messages: + msg390922
2020-07-09 17:45:53yselivanovsetmessages: + msg373410
2020-07-09 00:19:43cooperleessetmessages: + msg373364
2020-07-08 23:08:33cooperleessetkeywords: + patch
stage: patch review
pull_requests: + pull_request20556
2020-07-08 23:05:37yselivanovsetmessages: + msg373359
2020-07-08 21:29:56cooperleessetmessages: + msg373355
versions: + Python 3.10, - Python 3.8
2019-06-26 16:29:01cooperleessetmessages: + msg346650
2019-06-06 18:29:30fantixsetnosy: + fantix
2019-06-06 18:29:15yselivanovsetmessages: + msg344849
2019-06-06 17:55:13cooperleescreate