classification
Title: os.sendfile() on BSD, macOS don't return bytes sent on EINTR
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, josh.r, rosslagerwall
Priority: normal Keywords: patch

Created on 2019-03-30 16:47 by giampaolo.rodola, last changed 2019-04-12 16:11 by giampaolo.rodola.

Pull Requests
URL Status Linked Edit
PR 12807 open giampaolo.rodola, 2019-04-12 16:11
Messages (4)
msg339214 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2019-03-30 16:47
From "man sendfile" on both OSX and FreeBSD:

<<[EINTR] A signal interrupted sendfile() before it could be completed. If specified, the number of bytes successfully sent will be returned in *len.>>

Right now we catch EINTR and simply retry. Instead we should only retry is no bytes were sent, else we should return those bytes, similarly to what we do on EAGAIN and EBUSY:
https://github.com/python/cpython/blob/2438cdf0e932a341c7613bf4323d06b91ae9f1f1/Modules/posixmodule.c#L8907-L8917
msg339292 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2019-04-01 14:19
Wasn't the point of PEP475 that all EINTR returns would be explicitly handled by retrying rather than forcing the user to handle it? Seems like the correct solution is still to retry, but on OSX/FreeBSD we'd need to update the offset and count arguments to account for the data that has already been sent.
msg339511 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2019-04-05 18:44
sendfile() on BSD/OSX is complicated by the headers/trailers args. You'll have to take that into account in the retry logic, adding unnecessary complexity. Since sendfile() may already return fewer bytes than requested (e.g. non-blocking sockets or big files) it's just easier to return the bytes sent thus far (if any). I can work on a patch once I find some time.

> Wasn't the point of PEP475 that all EINTR returns would be explicitly handled by retrying rather than forcing the user to handle it?

From PEP475: <<[...] to relieve application code from the burden of doing so>>
msg339578 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2019-04-07 15:57
Right. So this is a hard problem for anyone to solve, and therefore os.sendfile should be the one solving it, not the caller of sendfile, right?
History
Date User Action Args
2019-04-12 16:11:35giampaolo.rodolasetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request12734
2019-04-07 15:57:12josh.rsetmessages: + msg339578
2019-04-05 18:44:05giampaolo.rodolasetmessages: + msg339511
2019-04-05 18:31:36terry.reedysettype: behavior
title: os.sendfile() on BSD and macOS does not return bytes sent on EINTR -> os.sendfile() on BSD, macOS don't return bytes sent on EINTR
2019-04-01 14:19:09josh.rsetnosy: + josh.r
messages: + msg339292
2019-03-30 19:39:42giampaolo.rodolasettitle: os.sendfile() on BSD OSs and macOS does not return bytes sent on EINTR -> os.sendfile() on BSD and macOS does not return bytes sent on EINTR
2019-03-30 19:39:10ned.deilysettitle: BSD/OSX: os.sendfile() does not return bytes sent on EINTR -> os.sendfile() on BSD OSs and macOS does not return bytes sent on EINTR
versions: - Python 3.6
2019-03-30 17:08:37giampaolo.rodolasetnosy: + rosslagerwall
2019-03-30 16:47:38giampaolo.rodolacreate