New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
os.sendfile can return EINVAL on Solaris #80791
Comments
Hi, We have several tests failing on Solaris due to the slightly different behavior of os.sendfile function. Sendfile on Solaris can raise EINVAL if offset is equal or bigger than the size of the file (Python expects that it will return 0 bytes sent in that case). I managed to patch EINVAL can also mean other things and so I guess I cannot just catch that errno and continue as with returned 0. Thanks |
Can you paste the traceback or are you able to reproduce the bug via a script? sendfile implementation is supposed to giveup if no data was sent on first call, so I suppose this happen later? If for any reason it turns out sendfile() is broken on Solaris or behave differently than on Linux I think I prefer to be safe than sorry and support it on Linux only. |
Here is a traceback from one failed test: test test_lib2to3 failed Traceback (most recent call last):
File "/tmp/Python-3.8.0a3/Lib/lib2to3/tests/test_refactor.py", line 228, in test_refactor_file_write_unchanged_file
self.check_file_refactoring(test_file, fixers=(),
File "/tmp/Python-3.8.0a3/Lib/lib2to3/tests/test_refactor.py", line 183, in check_file_refactoring
test_file = self.init_test_file(test_file)
File "/tmp/Python-3.8.0a3/Lib/lib2to3/tests/test_refactor.py", line 202, in init_test_file
shutil.copy(test_file, tmpdir)
File "/tmp/Python-3.8.0a3/Lib/shutil.py", line 401, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/tmp/Python-3.8.0a3/Lib/shutil.py", line 266, in copyfile
_fastcopy_sendfile(fsrc, fdst)
File "/tmp/Python-3.8.0a3/Lib/shutil.py", line 165, in _fastcopy_sendfile
raise err
File "/tmp/Python-3.8.0a3/Lib/shutil.py", line 145, in _fastcopy_sendfile
sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 22] Invalid argument: '/tmp/Python-3.8.0a3/Lib/lib2to3/tests/data/fixers/parrot_example.py' -> '/tmp/2to3-test_refactoruxve6nrz/parrot_example.py' I also have a script which can reproduce this (attached). It happens to me every time my offset is bigger or equal than the size of the file, no matter whether the file is empty or offset is bigger than the size right from the start, even when in a loop, sending in several parts. The attached example works on my Linux machine (outputs 12 and 0), but breaks on Solaris (12 and exception). As I have said before, it might be just that Python (specifically sendfile implementation) is not compiled correctly on Solaris and changes to define guards might fix that. |
I currently have no Solaris box to test this against and am currently short on time but I'm of the opinion that because of: > Sendfile on Solaris can raise EINVAL if offset is equal or bigger than the size ...and the fact that beta1 will happen soon it's safer to use sendfile() on Linux only. Googling for "solaris + sendfile" also raises suspicions that sendfile() may be broken on Solaris. shutil.copyfile() is too central to risk breaking it. We may also want to look at socket.sendfile() implementation and add tests for large files for both functions (socket.sendfile() and shutil.copyfile()). I'm adding that to my TODO list. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: