classification
Title: socket.send() fails to send large amount of bytes
Type: behavior Stage: resolved
Components: SSL Versions: Python 3.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: Kaulkwappe, christian.heimes, methane
Priority: normal Keywords:

Created on 2018-04-18 10:52 by Kaulkwappe, last changed 2018-04-18 13:45 by Kaulkwappe. This issue is now closed.

Messages (7)
msg315444 - (view) Author: (Kaulkwappe) Date: 2018-04-18 10:52
socket.setblocking(0)
socket.send(b'a' * 32 * 1024 * 1024)

In the example above socket.send() fails with this error:

Error in connection handler
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/websockets/server.py", line 81, in handler
    yield from self.ws_handler(self, path)
  File "/var/www/vhosts/.../app/sockets/core/Daemon.py", line 286, in websocketClientHandler
    self.api.connection.send(data)
  File "/usr/lib/python3.5/ssl.py", line 869, in send
    return self._sslobj.write(data)
  File "/usr/lib/python3.5/ssl.py", line 594, in write
    return self._sslobj.write(data)
ssl.SSLWantWriteError: The operation did not complete (write) (_ssl.c:1949)
msg315446 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-04-18 12:44
Isn't it an expected behavior of SSL with nonblocking socket?
What's wrong?
msg315447 - (view) Author: (Kaulkwappe) Date: 2018-04-18 12:57
Sorry for the misunderstanding, here is the complete example:

socket.setblocking(0)

try:
	buffer = socket.recv()

	if not buffer:
		break

except OSError:
	bytes_sent = socket.send(b'a' * 32 * 1024 * 1024)

In this case the socket is ready for sending data and should return the bytes that were actually sent. But when sending a large amount of bytes it every time fails with SSLWantWriteError exception.
msg315448 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-04-18 12:58
Inada is correct. This is the expected and documented behavior for non-blocking SSLSockets, see https://docs.python.org/3/library/ssl.html#ssl-nonblocking .
msg315449 - (view) Author: (Kaulkwappe) Date: 2018-04-18 13:12
But why does socket.send() throws an exception in this case only when sending a large amount of bytes? That would mean it is impossible to send large amount of bytes over SSL sockets as it would fail everytime.
msg315451 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-04-18 13:33
The same way as with any other non-blocking I/O system. You have to keep track how much data you have already sent and repeat non-blocking send() until you have succeeded. With TLS/SSL it's even more complex, because a send() also requires reading and a recv() also involves writing.

PS: bugs.python.org is an issue tracker, not a help forum.
msg315452 - (view) Author: (Kaulkwappe) Date: 2018-04-18 13:45
Thank you for your answer Christian. Indeed it seems a little more complex. As I worked with Non-TLS sockets before it looked like unexpected behaviour to me as on non-blocking sockets socket.send() would normally return 0 when no data was sent.

By the way this is the code I currently use:

Code:

	n = 0

	while 1:
		
		time.sleep(0.001)
		
		try:
		
			buffer = socket.recv(8192)
			
			if not buffer:
				break
			
		except OSError:
			print('{0}. try to send:'.format(n), len(blark), 'bytes')
			
			try:
				sent = socket.send(blark)
			except ssl.SSLWantWriteError:
				sent = 0
				n += 1
				
			print('Bytes sent:', sent)

		else:
			[...]
		
Result:

	1. try to send: 33554469 bytes
	Bytes sent: 0

	[...]

	137. try to send: 33554469 bytes
	Bytes sent: 0

	138. try to send: 33554469 bytes
	Bytes sent: 0

	139. try to send: 33554469 bytes
	Bytes sent: 33554469
History
Date User Action Args
2018-04-18 13:45:21Kaulkwappesetmessages: + msg315452
2018-04-18 13:33:53christian.heimessetstatus: open -> closed

messages: + msg315451
2018-04-18 13:12:16Kaulkwappesetstatus: closed -> open

messages: + msg315449
2018-04-18 12:58:47christian.heimessetstatus: open -> closed
type: crash -> behavior
messages: + msg315448

resolution: not a bug
stage: resolved
2018-04-18 12:57:22Kaulkwappesetmessages: + msg315447
2018-04-18 12:44:24methanesetnosy: + methane
messages: + msg315446
2018-04-18 10:52:55Kaulkwappecreate