classification
Title: Type: Improper NotADirectoryError when opening a file in a fake directory behavior Python 3.8
process
Status: Resolution: open danny87105, eryksun normal

Created on 2020-09-07 15:36 by danny87105, last changed 2020-09-11 23:23 by terry.reedy.

Messages (3)
msg376505 - (view) Author: Danny Lin (danny87105) Date: 2020-09-07 15:36
On Linux (tested on Ubuntu 16.04), if "/path/to/file" is an existing file, the code:

open('/path/to/file/somename.txt')

raises NotADirectoryError: [Errno 20] Not a directory: '/path/to/file/somename.txt'

On Windows, similar code:

open(r'C:\path\to\file\somename.txt')

raises FileNotFoundError: [Errno 2] No such file or directory: 'C:\\path\\chrome\\to\\file\\somename.txt'

I think the behavior on Linux is not correct. The user probably cares about the existence of the file to be opened, rather than whether its ancestor directories are valid.

OTOH, if NotADirectoryError should be raised, it should mention '/path/to/file' rather then '/path/to/file/somename.txt'. But what if '/path/to' or '/path' is actually a file? Should it be '/path/to' or '/path' instead for the same reason?
msg376529 - (view) Author: Eryk Sun (eryksun) * Date: 2020-09-07 21:14
> if NotADirectoryError should be raised, it should mention '/path/to/file'
> rather then '/path/to/file/somename.txt'.

POSIX specifies that C open() should set errno to ENOTDIR when an existing path prefix component is neither a directory nor a symlink to a directory [1]. What you propose isn't possible to implement reliably unless the filesystem is locked or readonly, so it should be handled by the application instead of by the system or standard libraries.

> FileNotFoundError: [Errno 2] No such file or directory:
> 'C:\\path\\chrome\\to\\file\\somename.txt'

Windows specifies that this case should fail as follows: "[i]f Link.File.FileType is not DirectoryFile, the operation MUST be failed with STATUS_OBJECT_PATH_NOT_FOUND" [2] (see Phase 6 in the linked pseudocode).

The Windows API maps this status code to ERROR_PATH_NOT_FOUND (3), which is distinct from ERROR_FILE_NOT_FOUND (2). However, the C runtime maps both of these system error codes to POSIX ENOENT. All isn't lost, however, because it also saves the OS error in _doserrno. io.FileIO could preset _doserrno to 0, and if it's non-zero after calling _wopen, use its value with PyErr_SetExcFromWindowsErrWithFilenameObject instead of calling PyErr_SetFromErrnoWithFilenameObject.

---

[1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html
[2] https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fsa/8ada5fbe-db4e-49fd-aef6-20d54b748e40
msg376562 - (view) Author: Danny Lin (danny87105) Date: 2020-09-08 10:24
I'm not so familiar about the spec. If such behavior is confirmed due to implementation difference across OSes, and it's also not desirable to change the mapping of the OS error to Python exception, we can simplify left it as-is.

However, this behavior difference can potentially cause cross-platform compatibility. For example, the code:

try:
open('/path/to/file/somename.txt')
except FileNotFoundError:
"""do something"""

would work on Windows but break on Linux (or POSIX?).

The current (3.8) description for exception NotADirectoryError is:

Raised when a directory operation (such as os.listdir()) is requested on something which is not a directory. Corresponds to errno ENOTDIR.

According to this, a user probably won't expect to receive a NotADirectoryError for open('/path/to/file/somename.txt'), as this doesn't seem like a directory operation at all, unless he is expert enough to know about the implication of errno ENOTDIR.

I think a note should at least be added to the documentation if we are not going to change the behavior.
History
Date User Action Args
2020-09-11 23:23:46terry.reedysettitle: Improper NotADirectoryError when opening a file under a fake directory -> Improper NotADirectoryError when opening a file in a fake directory
2020-09-08 10:24:50danny87105setmessages: + msg376562
2020-09-07 21:14:39eryksunsetnosy: + eryksun
messages: + msg376529
2020-09-07 15:36:37danny87105create