This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: socket makefile read-write discards received data
Type: behavior Stage: patch review
Components: IO Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ZackerySpytz, ammar2, kc, martin.panter, pravn, serhiy.storchaka
Priority: normal Keywords: 3.6regression, patch

Created on 2019-02-07 12:30 by pravn, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 11878 closed kc, 2019-02-15 16:30
PR 22535 open ZackerySpytz, 2020-10-04 05:50
Messages (5)
msg335017 - (view) Author: Palle Ravn (pravn) Date: 2019-02-07 12:30
Using socket.makefile in read-write mode had a bug introduced between version 3.6.6 and 3.6.7. The same bug is present in version 3.7.x.

The below code example will behave very differently between 3.6.6 and 3.6.7. It's based on the echo-server example from the docs.

import socket

HOST = '127.0.0.1'
PORT = 0

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind((HOST, PORT))

    print(f'Waiting for connection on port {s.getsockname()[1]}')
    s.listen(1)

    conn, addr = s.accept()
    print(f'Connected by {addr}')

    with conn:
        f = conn.makefile(mode='rw')

        while True:
            m = f.readline()
            print(f'msg: {m!r}')

            if not m:
                exit(0)

            f.write(m)
            f.flush()


Python 3.6.7:
Sending the string "Hello\nYou\n" will only print "Hello\n" and also only return "Hello\n" to the client.
Removing the lines with f.write(m) and f.flush() and both "Hello\n" and "You\n" will be returned to the client.
It's like the call to f.write() somehow empties the read buffer.

Python 3.6.6:
Sending "Hello\nYou\n" will return "Hello\n" and "You\n" to the client without any modifications to the above code.
msg335117 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2019-02-08 23:12
Looking over the changelog, my guess (untested) is this is caused by commit d6a283b3 for Issue 25862. That change looks like it drops the internal TextIOWrapper decoding buffer for each successful write.

I don't have the right version of Python to test with, but I expect this to also be broken without using a socket:

>>> f = TextIOWrapper(BufferedRWPair(BytesIO(b"Hello\nYou\n"), BytesIO()))
>>> f.readline()
'Hello\n'
>>> f.write(_)
6
>>> f.readline()  # Does this now return EOF?
'You\n'
msg335196 - (view) Author: Palle Ravn (pravn) Date: 2019-02-11 07:39
>>> f = TextIOWrapper(BufferedRWPair(BytesIO(b"Hello\nYou\n"), BytesIO()))
>>> f.readline()
'Hello\n'
>>> f.write(_)
6
>>> f.readline()  # Returns empty string
''
msg335197 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2019-02-11 07:52
Recreatable on master as well, also Martin your suspicion seems correct, reverting https://github.com/python/cpython/commit/23db935bcf258657682e66464bf8512def8af830 fixes it.
msg335631 - (view) Author: kc (kc) * Date: 2019-02-15 18:00
Added PR 11878, this will pass both this bug report and PR 3918 regression, the commit Ammar noted, it is an addition to this change.
History
Date User Action Args
2022-04-11 14:59:11adminsetgithub: 80109
2020-10-04 05:50:57ZackerySpytzsetnosy: + ZackerySpytz
pull_requests: + pull_request21538
2019-02-15 18:00:48kcsetnosy: + kc
messages: + msg335631
2019-02-15 16:30:00kcsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request11912
2019-02-11 07:52:57ammar2setversions: + Python 3.8
nosy: + serhiy.storchaka, ammar2

messages: + msg335197

stage: needs patch
2019-02-11 07:39:18pravnsetmessages: + msg335196
2019-02-08 23:12:59martin.pantersetkeywords: + 3.6regression
nosy: + martin.panter
messages: + msg335117

2019-02-07 12:30:32pravncreate