classification
Title: codecs.StreamReader doesn't work with nonblocking streams: TypeError: can't concat bytes to NoneType
Type: Stage: resolved
Components: Unicode Versions: Python 3.7, Python 3.4
process
Status: closed Resolution: duplicate
Dependencies: Superseder: The io module doesn't support non-blocking files
View: 13322
Assigned To: Nosy List: Fritz Reese, ezio.melotti, vstinner, yac
Priority: normal Keywords:

Created on 2015-07-03 15:12 by yac, last changed 2019-10-10 08:29 by vstinner. This issue is now closed.

Messages (4)
msg246186 - (view) Author: yac (yac) Date: 2015-07-03 15:12
File "/usr/lib64/python3.4/codecs.py", line 490, in read
    data = self.bytebuffer + newdata
TypeError: can't concat bytes to NoneType


            if size < 0:
                newdata = self.stream.read()
            else:
                newdata = self.stream.read(size)
            # decode bytes (those remaining from the last call included)
            data = self.bytebuffer + newdata

if self.stream is nonblocking, it's read will return None (py3, py2 raises IOError(EGAIN)).

Simple `if newdata is None: return None` should fix that I guess
msg246261 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-07-04 15:20
Use the io module instead using the open() function.
msg354298 - (view) Author: Fritz Reese (Fritz Reese) Date: 2019-10-09 21:33
This is still an issue on Linux in both 3.4 and 3.7 even when using io.open() as suggested by @vstinner:

>>> import io, os, fcntl
>>> r, w = os.pipe()
>>> fcntl(r, fcntl.F_SETFL, os.O_NONBLOCK)
0
>>> stream = io.open(r, 'rt')
>>> stream
<_io.TextIOWrapper name=X mode='rt' encoding='UTF-8'>
>>> stream.buffer
<_io.BufferedReader name=X>
>>> print(repr(stream.buffer.read()))
None
>>> stream.read()
Traceback (most recent call last):
  ...
  File ".../python3.7/codecs.py"..., in decode
    data = self.buffer + input
TypeError: can't concat NoneType to bytes

The error is present in at least 3.4 up to 3.7 where the underlying buffer.read() returns None which is not handled by the TextIOStream.
msg354340 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-10 08:29
I close this issue as a duplicate of bpo-13322.
History
Date User Action Args
2019-10-10 08:29:35vstinnersetstatus: open -> closed
superseder: The io module doesn't support non-blocking files
messages: + msg354340

resolution: duplicate
stage: resolved
2019-10-09 21:33:54Fritz Reesesetnosy: + Fritz Reese

messages: + msg354298
versions: + Python 3.7
2015-07-04 15:20:04vstinnersetmessages: + msg246261
2015-07-03 15:12:04yaccreate