classification
Title: misleading error message from shutil.copy()
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: shutil.copy raises IsADirectoryError when the directory does not actually exist
View: 43219
Assigned To: Nosy List: cedricvanrompay, eryksun, matrixise, mdk, serhiy.storchaka, vstinner, xtreak
Priority: normal Keywords:

Created on 2018-11-12 12:19 by cedricvanrompay, last changed 2021-03-17 07:30 by eryksun. This issue is now closed.

Messages (7)
msg329728 - (view) Author: Cédric Van Rompay (cedricvanrompay) Date: 2018-11-12 12:19
When calling `shutil.copy('file.txt', 'not_here/')`,
if directory `not_here/` does not exist,
the raised error is:

    IsADirectoryError: [Errno 21] Is a directory: 'not_here/'

If the intent of the user was to copy a file in a directory
but the user forgot to create the destination directory,
this can be very misleading,
as the error tends to indicate that the directory exists.

It happened to me and I was thinking
"yes it's a directory, then what?
that's exactly what I want,copy to this directory!"
when the problem was that I forgot to create the destination directory.

I would suggest to catch the `IsADirectoryError`
in shutil.copyfile() (at `open(dst, 'wb')`)
and raise instead an error saying something like
"destination is a directory AND this directory does not exists".
msg329730 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-11-12 13:25
Agreed this is confusing. Is this a Linux specific error? Trying this on Mac gives me a different error code and exception.

# Mac

$ ./python.exe
Python 3.8.0a0 (heads/master:cd449806fa, Nov 12 2018, 09:51:24)
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shutil
>>> shutil.copy('/tmp/a.py', 'Lib12/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/shutil.py", line 385, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/shutil.py", line 240, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: 'Lib12/'

# Ubuntu 

./python
Python 3.8.0a0 (heads/master:dce345c51a, Nov 12 2018, 13:01:05)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shutil
>>> shutil.copy('/tmp/a.py', 'Lib12/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/karthi/cpython/Lib/shutil.py", line 386, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/karthi/cpython/Lib/shutil.py", line 241, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
IsADirectoryError: [Errno 21] Is a directory: 'Lib12/'
msg329734 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-11-12 14:49
You have the issue with the built-in 'open' function.
msg329735 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-11-12 14:59
This error is specific to the C-API and not to Python,

here is an example.

#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>

int main(int argc, char **argv, char **environ) {
    int fd;
    fd = open("/tmp/toto/", O_CREAT);
    printf("file descriptor: %d\n", fd);
    printf(strerror(errno));
    close(fd);
    return 0;
}



./a.out
file descriptor: -1
Is a directory
msg329736 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-11-12 15:19
@vstinner & @serhiy

What do you think about this issue?

For the same POSIX syscall (open) we get 2 different values for the error, 2 on macOS and EISDIR with Linux.

Is a bug in Python or with the compliance of the operating system and the POSIX norm?

Thank you
msg329761 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2018-11-12 21:51
Using `cp` on Debian Buster I'm having a better error message:

    $ touch foo
    $ cp foo bar/
    cp: failed to access 'bar/': Not a directory

From copy.c (from Debian coreutils):

    /* Improve quality of diagnostic when a nonexistent dst_name
       ends in a slash and open fails with errno == EISDIR.  */
    if (dest_desc < 0 && dest_errno == EISDIR
        && *dst_name && dst_name[strlen (dst_name) - 1] == '/')
      dest_errno = ENOTDIR;
msg329856 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2018-11-13 17:43
The EINVAL error in Windows also needs improvement, but I don't what can be done after the fact. If there's a trailing slash when opening or creating a regular file, the NtCreateFile system call returns STATUS_OBJECT_INVALID_NAME. The Windows API maps this to ERROR_INVALID_NAME (123), which the CRT in turn maps to EINVAL (22). This error is too generic to handle. Even if the name ends in a slash, the error could be due to some other invalid character in the path (e.g. a common mistake is a '\t' or '\n' in a string literal). 

The problem could be addressed beforehand in shutil.copy by manually raising an exception if isdir() is false and the name has a trailing slash.
History
Date User Action Args
2021-03-17 07:30:11eryksunsetstatus: open -> closed
superseder: shutil.copy raises IsADirectoryError when the directory does not actually exist
resolution: duplicate
stage: resolved
2018-11-13 17:43:06eryksunsetnosy: + eryksun
messages: + msg329856
2018-11-12 21:51:08mdksetnosy: + mdk
messages: + msg329761
2018-11-12 15:19:40matrixisesetnosy: + vstinner, serhiy.storchaka
messages: + msg329736
2018-11-12 14:59:28matrixisesetmessages: + msg329735
2018-11-12 14:49:01matrixisesetnosy: + matrixise
messages: + msg329734
2018-11-12 13:25:16xtreaksetnosy: + xtreak
messages: + msg329730
2018-11-12 12:19:11cedricvanrompaycreate