Author Giovanni Pizzi
Recipients Giovanni Pizzi, ned.deily, ronaldoussoren
Date 2020-07-13.14:04:00
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1594649041.52.0.898762861377.issue41291@roundup.psfhosted.org>
In-reply-to
Content
Hello,

when creating deleting (with `os.remove`/`os.unlink`) a file and opening it in a different process at (almost) the same time, on Mac OS X relatively often I get a file that is empty instead of either a `FileNotFoundError` exception, or an open handle with the original content, i.e. at least one of the two operations (unlinking or opening) seems to be non-atomic.
(With empty I mean that if after I open the file (mode 'rb'), then `fhandle.read()` returns me b''.

More in particular, after quite some debugging I noticed that what happens is that if I stat the file descriptor, `st_ino` is zero.

This can reproduced very easily.
I set up a GitHub repository here: https://github.com/giovannipizzi/concurrent-delete-race-problem
with simple examples and tests (and both a Python implementation and a C implementation).
There are also GitHub Actions that run on Mac, Ubuntu and Windows, and apparently the problem exists only on Mac (and happens on all versions).

For completeness I attach also here a tar.gz with the two very short python files that just need be run in parallel - in my Mac OS X (macOS 10.14.6, 15" 2016 MacBook Pro, python 3.6) I get the error essentially at every run. 

Important: while much harder to reproduce, I can get the same error and behaviour also with the (I think equivalent) C code. 
Therefore, this seems to be more a problem with the Mac OS standard libraries?

My first question is: has anybody ever seen this problem? It seems to me quite easy to reproduce, but I'm surprised that I couldn't find any reference in the internet even after searching for various hours (maybe I used the wrong keywords?)

Second question: should this be reported directly to Apple?

Third question: Even if this is a bug, would it make sense to implement a patch in python that raises some exception if st_ino is zero when opening a file? Or am I simplifying too much and in some conditions st_ino=0 is valid on some types of mounted filesystems? Is there some other way to have a more atomic behaviour for this race condition in python?

Thanks a lot!
History
Date User Action Args
2020-07-13 14:04:01Giovanni Pizzisetrecipients: + Giovanni Pizzi, ronaldoussoren, ned.deily
2020-07-13 14:04:01Giovanni Pizzisetmessageid: <1594649041.52.0.898762861377.issue41291@roundup.psfhosted.org>
2020-07-13 14:04:01Giovanni Pizzilinkissue41291 messages
2020-07-13 14:04:00Giovanni Pizzicreate