This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author pmrv
Recipients PEAR, alexeicolin, giampaolo.rodola, gregory.p.smith, p.conesa.mingo, pmrv
Date 2022-03-13.15:14:53
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1647184494.12.0.719279084837.issue43743@roundup.psfhosted.org>
In-reply-to
Content
I hope you don't mind me necro posting, but I ran into this issue again and
have a small patch to solve it.

I attached an MWE that triggers the BlockingIOError reliably on ext4
filesystems in linux 4.12.14 and python 3.8.12.  Running under strace -e
sendfile gives the following output

# manually calling sendfile to check that it works
> sendfile(5, 4, [0] => [8388608], 8388608) = 8388608
# sendfile calls originating in shutil.copy
> sendfile(5, 4, [0] => [8388608], 8388608) = 8388608
> sendfile(5, 4, [8388608], 8388608)      = -1 EAGAIN (Resource temporarily unavailable)
> Shutil Failed!
> [Errno 11] Resource temporarily unavailable: '/cmmc/u/zora/scratch/sendfile_bug/tmpaqx2o4uj' -> '/cmmc/u/zora/scratch/sendfile_bug/tmpb8rzg8rg'
> +++ exited with 0 +++

This shows that the first call to sendfile actually copies the whole file and
the EAGAIN is only triggered on the second, unnecessary, call.  I have tested
with a small C program that it's triggered whenever sendfile's offset + count
exceeds the file size of in_fd.  This is weird behaviour on the kernels side
that seems to have changed in newer kernel versions (issue is not present e.g.
on my 5.16.12 laptop).

Anyways my patch makes that second call not appear by keeping track of the file
size and the bytes written so far.  It's against the current python main
branch, but if I see correctly this part hasn't changed in years.  I have
checked the error is not thrown when the patch is applied.

(I can only attach one file, so patch is attached in a new one.)
History
Date User Action Args
2022-03-13 15:14:54pmrvsetrecipients: + pmrv, gregory.p.smith, giampaolo.rodola, alexeicolin, p.conesa.mingo, PEAR
2022-03-13 15:14:54pmrvsetmessageid: <1647184494.12.0.719279084837.issue43743@roundup.psfhosted.org>
2022-03-13 15:14:54pmrvlinkissue43743 messages
2022-03-13 15:14:53pmrvcreate