This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Re: Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range
Type: behavior Stage: resolved
Components: asyncio Versions: Python 3.7
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, ben.brown, christian.heimes, m_emori, maayank, yselivanov
Priority: normal Keywords: patch

Created on 2020-08-20 07:47 by m_emori, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
sslproto.py.patch m_emori, 2020-08-20 07:47 exclude data operations on the array (_write_backlog)
Messages (3)
msg375689 - (view) Author: Mototsugu Emori (m_emori) Date: 2020-08-20 07:47
When using SSL with websocket, _process_write_backlog function throws IndexError.

---------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/websockets/protocol.py", line 827, in transfer_data
    message = await self.read_message()
  File "/usr/local/lib/python3.7/site-packages/websockets/protocol.py", line 895, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
  File "/usr/local/lib/python3.7/site-packages/websockets/protocol.py", line 971, in read_data_frame
    frame = await self.read_frame(max_size)
  File "/usr/local/lib/python3.7/site-packages/websockets/protocol.py", line 1051, in read_frame
    extensions=self.extensions,
  File "/usr/local/lib/python3.7/site-packages/websockets/framing.py", line 105, in read
    data = await reader(2)
  File "/usr/local/lib/python3.7/asyncio/streams.py", line 679, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/local/lib/python3.7/asyncio/streams.py", line 473, in _wait_for_data
    await self._waiter
  File "/usr/local/lib/python3.7/asyncio/sslproto.py", line 689, in _process_write_backlog
    del self._write_backlog[0]
IndexError: deque index out of range
---------------------------------------------------------------

Websocket has a keepalive feature and periodically pings the server.
If ping sending process occurs at the same time when sending data, the data operation to the array (_write_backlog) will be duplicated and the error will occur.

---------------------------------------------------------------
asyncio/sslproto.py
---------------------------------------------------------------
class SSLProtocol(protocols.Protocol)
    def _write_appdata(self, data):
        self._write_backlog.append((data, 0)) <--- append data to _write_backlog
        self._write_buffer_size += len(data)
        self._process_write_backlog()

    def _process_write_backlog(self):
        del self._write_backlog[0]  <--- !!!ERROR!!! IndexError: deque index out of range
---------------------------------------------------------------

I made a patch(sslproto.sy.patch) to exclude data operations on the array (_write_backlog).
Could you check it?

Environments:
  debian 4.19.46
  Python 3.7.3
  OpenSSL 1.1.0l  10 Sep 2019

Reference:
  Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range
  https://bugs.python.org/issue37226
msg375694 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-08-20 08:52
This also looks like a concurrency issue in the websocket package. Also see #41597
msg375837 - (view) Author: Mototsugu Emori (m_emori) Date: 2020-08-24 08:28
Thank you for your reply.
I understand that websocket cannot be used in multiple threads.
I close the issue.
History
Date User Action Args
2022-04-11 14:59:34adminsetgithub: 85762
2020-08-24 08:28:59m_emorisetstatus: open -> closed

messages: + msg375837
stage: resolved
2020-08-20 08:52:47christian.heimessetmessages: + msg375694
2020-08-20 07:47:51m_emoricreate