On Linux (tested on Ubuntu 16.04), if "/path/to/file" is an existing file, the code:

    open('/path/to/file/somename.txt')

raises NotADirectoryError: [Errno 20] Not a directory: '/path/to/file/somename.txt'

On Windows, similar code:

    open(r'C:\path\to\file\somename.txt')

raises FileNotFoundError: [Errno 2] No such file or directory: 'C:\\path\\chrome\\to\\file\\somename.txt'

I think the behavior on Linux is not correct. The user probably cares about the existence of the file to be opened, rather than whether its ancestor directories are valid.

OTOH, if NotADirectoryError should be raised, it should mention '/path/to/file' rather then '/path/to/file/somename.txt'. But what if '/path/to' or '/path' is actually a file? Should it be '/path/to' or '/path' instead for the same reason?

> if NotADirectoryError should be raised, it should mention '/path/to/file' 
> rather then '/path/to/file/somename.txt'.

POSIX specifies that C open() should set errno to ENOTDIR when an existing path prefix component is neither a directory nor a symlink to a directory [1]. What you propose isn't possible to implement reliably unless the filesystem is locked or readonly, so it should be handled by the application instead of by the system or standard libraries.

> FileNotFoundError: [Errno 2] No such file or directory: 
> 'C:\\path\\chrome\\to\\file\\somename.txt'

Windows specifies that this case should fail as follows: "[i]f Link.File.FileType is not DirectoryFile, the operation MUST be failed with STATUS_OBJECT_PATH_NOT_FOUND" [2] (see Phase 6 in the linked pseudocode). 

The Windows API maps this status code to ERROR_PATH_NOT_FOUND (3), which is distinct from ERROR_FILE_NOT_FOUND (2). However, the C runtime maps both of these system error codes to POSIX ENOENT. All isn't lost, however, because it also saves the OS error in _doserrno. io.FileIO could preset _doserrno to 0, and if it's non-zero after calling _wopen, use its value with PyErr_SetExcFromWindowsErrWithFilenameObject instead of calling PyErr_SetFromErrnoWithFilenameObject.

---

[1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html
[2] https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fsa/8ada5fbe-db4e-49fd-aef6-20d54b748e40

I'm not so familiar about the spec. If such behavior is confirmed due to implementation difference across OSes, and it's also not desirable to change the mapping of the OS error to Python exception, we can simplify left it as-is.

However, this behavior difference can potentially cause cross-platform compatibility. For example, the code:

    try:
        open('/path/to/file/somename.txt')
    except FileNotFoundError:
        """do something"""

would work on Windows but break on Linux (or POSIX?).

The current (3.8) description for exception NotADirectoryError is:

    Raised when a directory operation (such as os.listdir()) is requested on something which is not a directory. Corresponds to errno ENOTDIR.

According to this, a user probably won't expect to receive a NotADirectoryError for open('/path/to/file/somename.txt'), as this doesn't seem like a directory operation at all, unless he is expert enough to know about the implication of errno ENOTDIR.

I think a note should at least be added to the documentation if we are not going to change the behavior.

The documentation for open says "If the file cannot be opened, an OSError is raised."

NotADirectoryError and FileNotFoundError are both OSErrors.

So the correct way to write Danny's code snippet, according to the documentation, is:

    try:
        open('/path/to/file/somename.txt')
    except OSError:
        """do something"""

By writing "except FileNotFoundError:", the intention is to catch an error when the file being opened is not found, and don't catch an error for other cases, such as an existing file without adequate permission. Writing "except OSError:" catches just too much cases including unwanted ones.

As they are not equivalent, you cannot say that the latter is the "correct" way for the former.

And you totally omitted the argument for the inadequate documentation for NotADirectoryError. It says NotADirectoryError is raises when a DIRECTORY operation is requested, and does not cover the case of opening a file.

Hi Danny, 

I'm not saying that OSError and FileNotFoundError are equivalent. I'm saying that the open() API, as documented, raises OSError when opening the file fails.

The way to check whether a file exists is to use os.path.exists(path) or os.path.isfile(path). 

I don't quite follow your last point - opening a file is indeed a file operation, but it contains within it a directory operation (finding the file).

I don't think a general developer would expect that open('/path/to/file/somename.txt') implies a directory operation, and it also doesn't on Windows.

I suggest that a further notice be added to NotADirectoryError, such as:

    Raised when a directory operation (such as os.listdir()) is requested on something which is not a directory. Corresponds to errno ENOTDIR. In some filesystem such as POSIX, NotADirectoryError (ENOTDIR) is raised when attempting to open a path whose ancestor is not a directory.

> I don't think a general developer would expect that open('/path/to/file/somename.txt') implies a directory operation, and it also doesn't on Windows.


Really? It's not obvious that finding a file would involve directory operations?

In what sense does it even matter whether you expect a directory operation to happen? The contract is that if the open() fails you get an OSError. The documentation doesn't say which OSError, and that is in fact a platform-specific implementation detail.

> Really? It's not obvious that finding a file would involve directory operations?

Not in some senses, and that's possibly why Windows does not raise NotADirectoryError in such case.

I agree that it's a platform-specific implementation detail. However, there are lots of platform-specific implementation details written in documentation elsewhere already, especially OS related modules such as os and os.path. I don't think mentioning platform-specific implementation details for subclasses of OSError would be any less reasonable than that.

Adding such notice for NotADirectoryError helps people who want a EAFP style code for nonexistent file by preventing them getting trapped writing a cross-platform code.

Regarding documentation, builtin open() and the io and os modules generally do not provide information about platform-specific errors. But documenting the behavior for the NotADirectoryError exception itself may be useful considering it applies to many POSIX functions that access filepaths, such as stat, open, mkdir, rmdir, unlink, and rename.

Regarding behavior, I don't see anything reasonable that can or should be done for POSIX. In Windows, it should be possible to know whether FileNotFoundError is due to a bad path prefix (ERROR_PATH_NOT_FOUND, 3) or a missing file (ERROR_FILE_NOT_FOUND, 2), but io.FileIO currently only uses standard C errno, which maps both of these cases to ENOENT. So one behavior that *can* be fixed in this situation is to get the actual Windows error code from MSVC _doserrno, such that the raised FileNotFoundError would have the relevant Windows error code in its winerror attribute.

I've put up a PR that expands the docs for NotADirectoryError here: 

https://github.com/python/cpython/pull/27471/files

> I've put up a PR that expands the docs for NotADirectoryError here: 
> https://github.com/python/cpython/pull/27471/files

Thank you.

Wouldn't it be more clear if an example is provided? Like:

(e.g. `/path/to` does not exist when running `open('/path/to/file')`)

Danny: that would be inconsistent with the rest of the doc, it's written as a reference list for exceptions and none of them have examples currently.. It also seems fairly clear without an example.

(in fact, there is one example (for f-string exception) but the overall pattern is to just have reference type descriptions)

@Andrei Kulakov: I was commenting on the previous version. The revised version (f2ae30b0de3c4ba1f16fc2a430cf22b447c062ed) seems ok to me.

Another question: would it be better if we add "on some platforms" for the part that the error may raise on a file operation?

Danny: then it would be probably more useful to say "On POSIX systems, it will be raised when ...".

I'm ambivalent on whether this is needed.

Eryk: wdyt?

@Andrei Kulakov: That statement is mainly for illustration that such behavior may vary across platforms. I use a rather vague statement as I'm not totally sure it applies if (and only if) running on POSIX. A more accurate statement is welcome as long as it's proven true.

I'd prefer a generic wording regarding the platform, and overall simpler phrasing without examples: "On some platforms, it may also be raised if an operation attempts to open or traverse a non-directory file as if it were a directory." 

The latter doesn't apply to Windows generally. Attempting to traverse a non-directory fails with ERROR_PATH_NOT_FOUND (mapped to C errno ENOENT), and attempting to open a non-directory with a path that has a trailing slash fails with ERROR_INVALID_NAME (mapped to C errno EINVAL).

Eryk: sounds good, updated the PR.

It should be emphasized that it may happen on a **file operation**... So at least be something like: "On some platforms, it may also be raised if a file operation involves an attempt to open or traverse a non-directory file as if it were a directory."

> It should be emphasized that it may happen on a **file operation**

I think it's adequately covered by "attempts to open or traverse a non-directory file". The reader should know that opening "/path/to/file/somename.txt" requires traversing the components in the path. So if "file" isn't a directory, raising NotADirectoryError should be expected in POSIX.

---

If someone can verify the behavior on common non-Linux POSIX systems such as macOS, FreeBSD, and OpenBSD, then the wording could be narrowed down to "on most POSIX platforms" instead of "on some platforms". 

For example, given "spam" is a regular file in the current directory, check os.open('spam', os.O_DIRECTORY); os.open('spam/', 0); and os.open('spam/eggs', 0).

Eryk: On MacOS, I get Not a directory error on all 3:

NotADirectoryError: [Errno 20] Not a directory: 'spam'

NotADirectoryError: [Errno 20] Not a directory: 'spam/'

NotADirectoryError: [Errno 20] Not a directory: 'spam/eggs'

> On MacOS, I get Not a directory error on all 3

Confirmation on Linux and macOS suffices for "most POSIX systems", based on market share for desktops and servers. It would be nice to also confirm this for FreeBSD, AIX, HP-UX, and Solaris, but I suppose it isn't necessary. A system that strictly conforms to POSIX will behave the same for those operations.

New changeset f7c23a99cd4f8179b6ba2cffaeb78b852c0f6488 by andrei kulakov in branch 'main':
bpo-41737: expand doc for NotADirectoryError (GH-27471)
https://github.com/python/cpython/commit/f7c23a99cd4f8179b6ba2cffaeb78b852c0f6488

New changeset b5f026112768eb0a06622263bdea86d7d85981c5 by Miss Islington (bot) in branch '3.9':
bpo-41737: expand doc for NotADirectoryError (GH-27471) (GH-27577)
https://github.com/python/cpython/commit/b5f026112768eb0a06622263bdea86d7d85981c5

New changeset 84494db41902774ea6ac72e5b308b429850bbf71 by Miss Islington (bot) in branch '3.10':
bpo-41737: expand doc for NotADirectoryError (GH-27471)
https://github.com/python/cpython/commit/84494db41902774ea6ac72e5b308b429850bbf71