Message373606
Hello,
when creating deleting (with `os.remove`/`os.unlink`) a file and opening it in a different process at (almost) the same time, on Mac OS X relatively often I get a file that is empty instead of either a `FileNotFoundError` exception, or an open handle with the original content, i.e. at least one of the two operations (unlinking or opening) seems to be non-atomic.
(With empty I mean that if after I open the file (mode 'rb'), then `fhandle.read()` returns me b''.
More in particular, after quite some debugging I noticed that what happens is that if I stat the file descriptor, `st_ino` is zero.
This can reproduced very easily.
I set up a GitHub repository here: https://github.com/giovannipizzi/concurrent-delete-race-problem
with simple examples and tests (and both a Python implementation and a C implementation).
There are also GitHub Actions that run on Mac, Ubuntu and Windows, and apparently the problem exists only on Mac (and happens on all versions).
For completeness I attach also here a tar.gz with the two very short python files that just need be run in parallel - in my Mac OS X (macOS 10.14.6, 15" 2016 MacBook Pro, python 3.6) I get the error essentially at every run.
Important: while much harder to reproduce, I can get the same error and behaviour also with the (I think equivalent) C code.
Therefore, this seems to be more a problem with the Mac OS standard libraries?
My first question is: has anybody ever seen this problem? It seems to me quite easy to reproduce, but I'm surprised that I couldn't find any reference in the internet even after searching for various hours (maybe I used the wrong keywords?)
Second question: should this be reported directly to Apple?
Third question: Even if this is a bug, would it make sense to implement a patch in python that raises some exception if st_ino is zero when opening a file? Or am I simplifying too much and in some conditions st_ino=0 is valid on some types of mounted filesystems? Is there some other way to have a more atomic behaviour for this race condition in python?
Thanks a lot! |
|
Date |
User |
Action |
Args |
2020-07-13 14:04:01 | Giovanni Pizzi | set | recipients:
+ Giovanni Pizzi, ronaldoussoren, ned.deily |
2020-07-13 14:04:01 | Giovanni Pizzi | set | messageid: <1594649041.52.0.898762861377.issue41291@roundup.psfhosted.org> |
2020-07-13 14:04:01 | Giovanni Pizzi | link | issue41291 messages |
2020-07-13 14:04:00 | Giovanni Pizzi | create | |
|