Title: some methods of uninitialized io.IncrementalNewlineDecoder objects raise SystemError
Type: behavior Stage: patch review
Components: IO Versions: Python 3.9, Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Oren Milman, ZackerySpytz, benjamin.peterson, cheryl.sabella, serhiy.storchaka, stutzbach
Priority: normal Keywords: patch

Created on 2017-10-06 18:09 by Oren Milman, last changed 2020-02-24 06:50 by ZackerySpytz.

Pull Requests
URL Status Linked Edit
PR 3913 closed Oren Milman, 2017-10-07 09:10
PR 18640 open ZackerySpytz, 2020-02-24 06:50
Messages (6)
msg303842 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-10-06 18:09
Given an uninitialized IncrementalNewlineDecoder:
uninitialized = io.IncrementalNewlineDecoder.__new__(io.IncrementalNewlineDecoder)

each of the following calls would raise a SystemError ('null argument to
internal routine'):
uninitialized.setstate((b'foo', 0))

In contrast, the following call would raise a ValueError
('IncrementalNewlineDecoder.__init__ not called'):

ISTM that getstate(), setstate(), and reset() should have the same behavior as
decode(). (Though i think that including the actual type name in the error
message would be better, as it could be a subclass of IncrementalNewlineDecoder).
msg303847 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-10-06 19:40
Is IncrementalNewlineDecoder subclassable?
msg303853 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-10-06 20:33
Yes, although i don't know if there are usecases for that.
msg303860 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-10-06 21:00
There is other issues with IncrementalNewlineDecoder.__init__ -- it leaks references when called repeatedly.

The simplest solution of both issues will be moving the initialization to the new method. But this class looks designed for subclassing, and this can break subclasses that change arguments before passing them to the superclass' constructor.
msg303874 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-10-07 10:16
With regard to refleaks in __init__() methods, i started looking for similar refleaks
in the codebase, and hope to open an issue to fix them soon.
msg361625 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2020-02-08 11:20
The patch for this issue had some outstanding questions/changes to be made before being merged, but now a new PR needs to be opened to replace the original one.  Anyone who is interested can pick this up, but please only open one PR for this and please also credit the original author as per the dev guide.  Thank you!
Date User Action Args
2020-02-24 06:50:50ZackerySpytzsetnosy: + ZackerySpytz
pull_requests: + pull_request18006
2020-02-08 11:20:18cheryl.sabellasetnosy: + cheryl.sabella

messages: + msg361625
versions: + Python 3.8, Python 3.9, - Python 2.7, Python 3.6, Python 3.7
2017-10-07 14:28:38serhiy.storchakasetnosy: + benjamin.peterson, stutzbach

versions: + Python 2.7, Python 3.6
2017-10-07 10:16:13Oren Milmansetmessages: + msg303874
2017-10-07 09:10:48Oren Milmansetkeywords: + patch
stage: patch review
pull_requests: + pull_request3886
2017-10-06 21:00:09serhiy.storchakasetmessages: + msg303860
2017-10-06 20:33:32Oren Milmansetmessages: + msg303853
2017-10-06 19:40:21serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg303847
2017-10-06 18:09:48Oren Milmancreate