He's still going to get an error using your code, Christian.  But if he knows that the file being gone is OK, he can catch and ignore the error.  Having exit do the unlink wouldn't help him; in that case he'd have to wrap the whole 'with' clause in a try/except.

So I think delete=False and handling the error "manually" is the correct solution here, since I doubt we want to add a "already_deleted_ok=False" flag to the API for this limited use case that can be handled with delete=False.

I vote for closing this as rejected.