URL |
Status |
Linked |
Edit |
PR 27049 |
merged |
andrei.avk,
2021-07-07 02:29
|
|
PR 27081 |
merged |
miss-islington,
2021-07-10 03:47
|
|
PR 27082 |
merged |
miss-islington,
2021-07-10 03:48
|
|
PR 27257 |
merged |
kulikjak,
2021-07-20 10:18
|
|
PR 27267 |
merged |
miss-islington,
2021-07-20 18:16
|
|
PR 27268 |
merged |
miss-islington,
2021-07-20 18:16
|
|
PR 28507 |
merged |
miss-islington,
2021-09-21 21:53
|
|
PR 28508 |
merged |
miss-islington,
2021-09-21 21:53
|
|
msg386932 - (view) |
Author: Jeremy Pinto (jerpint) |
Date: 2021-02-14 05:16 |
Issue: If you try to copy a file to a directory that doesn't exist using shutil.copy, a IsADirectory error is raised saying the directory exists.
This issue is actually caused when `open(not_a_dir, 'wb') is called on a non-existing dir.
Expected behaviour: Should instead raise NotADirectoryError
-----------------------------
Steps to reproduce:
[nav] In [1]: import os
...: from pathlib import Path
...: from shutil import copy
...:
...: tmp_file = '/tmp/some_file.txt'
...: Path(tmp_file).touch()
...: nonexistent_dir = 'not_a_dir/'
...: assert not os.path.exists(nonexistent_dir)
...: copy(tmp_file, nonexistent_dir)
---------------------------------------------------------------------------
IsADirectoryError Traceback (most recent call last)
<ipython-input-2-b0e0ec4f4875> in <module>
7 nonexistent_dir = 'not_a_dir/'
8 assert not os.path.exists(nonexistent_dir)
----> 9 copy(tmp_file, nonexistent_dir)
~/miniconda3/lib/python3.7/shutil.py in copy(src, dst, follow_symlinks)
243 if os.path.isdir(dst):
244 dst = os.path.join(dst, os.path.basename(src))
--> 245 copyfile(src, dst, follow_symlinks=follow_symlinks)
246 copymode(src, dst, follow_symlinks=follow_symlinks)
247 return dst
~/miniconda3/lib/python3.7/shutil.py in copyfile(src, dst, follow_symlinks)
119 else:
120 with open(src, 'rb') as fsrc:
--> 121 with open(dst, 'wb') as fdst:
122 copyfileobj(fsrc, fdst)
123 return dst
IsADirectoryError: [Errno 21] Is a directory: 'not_a_dir/'
|
msg386933 - (view) |
Author: Jeremy Pinto (jerpint) |
Date: 2021-02-14 05:28 |
In fact, the issue seems to be coming from open() itself when opening a non-existent directory in write mode:
[nav] In [1]: import os
...: nonexixstent_dir = 'not_a_dir/'
...: assert not os.path.exists(nonexixstent_dir)
...: with open(nonexixstent_dir, 'wb') as fdst:
...: pass
---------------------------------------------------------------------------
IsADirectoryError Traceback (most recent call last)
<ipython-input-1-73d4010d6f34> in <module>
2 dir_path = 'not_a_dir/'
3 assert not os.path.exists(nonexixstent_dir)
----> 4 with open(nonexixstent_dir, 'wb') as fdst:
5 pass
IsADirectoryError: [Errno 21] Is a directory: 'not_a_dir/'
|
msg386935 - (view) |
Author: Eryk Sun (eryksun) *  |
Date: 2021-02-14 06:49 |
> IsADirectoryError: [Errno 21] Is a directory: 'not_a_dir/'
The trailing slash forces the OS to handle "not_a_dir" as a directory [1].
A pathname that contains at least one non- <slash> character and that
ends with one or more trailing <slash> characters shall not be resolved
successfully unless the last pathname component before the trailing
<slash> characters names an existing directory or a directory entry
that is to be created for a directory immediately after the pathname is
resolved.
Mode "w" corresponds to low-level POSIX open() flags O_CREAT | O_TRUNC | O_WRONLY. If write access is requested for a directory, the open() system call must fail with EISDIR [2].
[EISDIR]
The named file is a directory and oflag includes O_WRONLY or O_RDWR,
or includes O_CREAT without O_DIRECTORY.
In most cases, opening a directory with O_CREAT also fails with E_ISDIR. POSIX does permit an implementation to create a directory with O_CREAT | O_DIRECTORY. In Linux, however, O_CREAT always creates a regular file, regardless of O_DIRECTORY, so open(pathname, O_CREAT | flags) always fails with EISDIR when pathname is an existing directory or names a directory by way of a trailing slash.
---
[1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_13
[2] https://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html
|
msg387134 - (view) |
Author: Eryk Sun (eryksun) *  |
Date: 2021-02-17 02:02 |
I left this open in case someone wants to modify shutil.copy() and shutil.copy2() to raise a less misleading exception when `dst` names a non-existing directory such as 'not_a_dir/'. Failing with IsADirectoryError (errno EISDIR) is confusing since shutil.copy() and shutil.copy2() do support a destination directory.
Note that in Windows this case fails with EINVAL, which is at least less misleading than EISDIR. The EINVAL error is based on WinAPI ERROR_INVALID_NAME (123). CreateFileW() can create a directory if passed particular parameters, in which case a trailing slash is allowed. Otherwise it fails with ERROR_INVALID_NAME (123), unless it's an open-existing disposition, in which case it fails with ERROR_FILE_NOT_FOUND (2). This is specified in [MS-FSA] 2.1.5.1 [1]:
Phase 6 -- Location of file (final path component):
Search ParentFile.DirectoryList for a Link where Link.Name or
Link.ShortName matches FileNameToOpen. If such a link is found:
Set File = Link.File.
Set Open.File to File.
Set Open.Link to Link.
Else:
If (CreateDisposition == FILE_OPEN || CreateDisposition ==
FILE_OVERWRITE), the operation MUST be failed with
STATUS_OBJECT_NAME_NOT_FOUND.
Phase 7 -- Type of stream to open:
If PathName contains a trailing backslash:
If StreamTypeToOpen is DataStream or
CreateOptions.FILE_NON_DIRECTORY_FILE is TRUE, the operation
MUST be failed with STATUS_OBJECT_NAME_INVALID.
NTAPI STATUS_OBJECT_NAME_NOT_FOUND and STATUS_OBJECT_NAME_INVALID map to WinAPI ERROR_FILE_NOT_FOUND and ERROR_INVALID_NAME.
---
[1] https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fsa/8ada5fbe-db4e-49fd-aef6-20d54b748e40
|
msg397060 - (view) |
Author: Andrei Kulakov (andrei.avk) *  |
Date: 2021-07-07 03:23 |
I've opened the PR here:
https://github.com/python/cpython/pull/27049/files
.. but can someone trigger builds for other OSes like Aix, FreeBSD, etc? I'm not sure what errors they would use for this case so the unit test may have to be updated to skip some of them.
|
msg397061 - (view) |
Author: Andrei Kulakov (andrei.avk) *  |
Date: 2021-07-07 03:26 |
I'm not sure if shutil module docs should be updated or not, as this change applies to a narrow corner case.
|
msg397119 - (view) |
Author: Andrei Kulakov (andrei.avk) *  |
Date: 2021-07-07 21:18 |
Note that someone else ran into this confusion here: https://bugs.python.org/issue43153 .
|
msg397235 - (view) |
Author: Gregory P. Smith (gregory.p.smith) *  |
Date: 2021-07-10 03:47 |
New changeset 248173cc0483a9ad9261353302f1234cf9eb2ebe by andrei kulakov in branch 'main':
bpo-43219: shutil.copyfile, raise a less confusing exception instead of IsADirectoryError (GH-27049)
https://github.com/python/cpython/commit/248173cc0483a9ad9261353302f1234cf9eb2ebe
|
msg397236 - (view) |
Author: miss-islington (miss-islington) |
Date: 2021-07-10 04:07 |
New changeset 1577259cc51d0d46ad676798ce0a130039acf956 by Miss Islington (bot) in branch '3.10':
bpo-43219: shutil.copyfile, raise a less confusing exception instead of IsADirectoryError (GH-27049)
https://github.com/python/cpython/commit/1577259cc51d0d46ad676798ce0a130039acf956
|
msg397237 - (view) |
Author: miss-islington (miss-islington) |
Date: 2021-07-10 04:14 |
New changeset c89f0b2587eb0b16175a0bbb12d0b86314ff9320 by Miss Islington (bot) in branch '3.9':
[3.9] bpo-43219: shutil.copyfile, raise a less confusing exception instead of IsADirectoryError (GH-27049) (GH-27082)
https://github.com/python/cpython/commit/c89f0b2587eb0b16175a0bbb12d0b86314ff9320
|
msg397865 - (view) |
Author: Jakub Kulik (kulikjak) * |
Date: 2021-07-20 10:17 |
On Solaris (I checked this on Oracle and SmartOS), the error is:
NotADirectoryError: [Errno 20] Not a directory: 'not_a_dir/'
which I think belongs to the 'errors are not confusing' category with Windows and macOS.
|
msg397893 - (view) |
Author: Gregory P. Smith (gregory.p.smith) *  |
Date: 2021-07-20 18:16 |
New changeset 6564656495d456a1bcc1aaa06abfc696209f37b2 by Jakub Kulík in branch 'main':
bpo-43219: skip Solaris in the test as well (GH-27257)
https://github.com/python/cpython/commit/6564656495d456a1bcc1aaa06abfc696209f37b2
|
msg397897 - (view) |
Author: miss-islington (miss-islington) |
Date: 2021-07-20 18:45 |
New changeset dae4928dd07109db69e090b1c8193a023ce695cd by Miss Islington (bot) in branch '3.9':
[3.9] bpo-43219: skip Solaris in the test as well (GH-27257) (GH-27267)
https://github.com/python/cpython/commit/dae4928dd07109db69e090b1c8193a023ce695cd
|
msg397898 - (view) |
Author: miss-islington (miss-islington) |
Date: 2021-07-20 18:53 |
New changeset 574da4633b44b4048f74c93da496ed2a3ead99dd by Miss Islington (bot) in branch '3.10':
[3.10] bpo-43219: skip Solaris in the test as well (GH-27257) (GH-27268)
https://github.com/python/cpython/commit/574da4633b44b4048f74c93da496ed2a3ead99dd
|
msg397901 - (view) |
Author: Andrei Kulakov (andrei.avk) *  |
Date: 2021-07-20 19:03 |
Thanks for reporting Jakub, and for patching Gregory!
|
msg402022 - (view) |
Author: Alex Grund (Alex Grund) |
Date: 2021-09-17 09:28 |
The changelog wrongfully links to https://bugs.python.org/issue41928 instead of this issue.
Also the fix introduced a regression: Trying to copy a directory now raises a FileNotFoundError
|
msg402373 - (view) |
Author: Łukasz Langa (lukasz.langa) *  |
Date: 2021-09-21 21:53 |
New changeset b7eac52b466f697d3e89f47508e0df0196a98970 by andrei kulakov in branch 'main':
bpo-45234: Fix FileNotFound exception raised instead of IsADirectoryError in shutil.copyfile() (GH-28421)
https://github.com/python/cpython/commit/b7eac52b466f697d3e89f47508e0df0196a98970
|
msg402386 - (view) |
Author: Łukasz Langa (lukasz.langa) *  |
Date: 2021-09-21 22:14 |
New changeset 41d48bc038b254cc4a78a2d840097196b9545a84 by Miss Islington (bot) in branch '3.10':
bpo-45234: Fix FileNotFound exception raised instead of IsADirectoryError in shutil.copyfile() (GH-28421) (GH-28508)
https://github.com/python/cpython/commit/41d48bc038b254cc4a78a2d840097196b9545a84
|
msg402394 - (view) |
Author: Łukasz Langa (lukasz.langa) *  |
Date: 2021-09-21 22:21 |
New changeset 09390c837a0bf73e213db2fbde34d756fa77a837 by Miss Islington (bot) in branch '3.9':
bpo-45234: Fix FileNotFound exception raised instead of IsADirectoryError in shutil.copyfile() (GH-28421) (GH-28507)
https://github.com/python/cpython/commit/09390c837a0bf73e213db2fbde34d756fa77a837
|
msg403155 - (view) |
Author: Pablo Galindo Salgado (pablogsal) *  |
Date: 2021-10-04 19:18 |
New changeset e0b61b28641bdd20cfeff6d9878f1318b711ca19 by Pablo Galindo (Miss Islington (bot)) in branch '3.10':
bpo-45234: Fix FileNotFound exception raised instead of IsADirectoryError in shutil.copyfile() (GH-28421) (GH-28508)
https://github.com/python/cpython/commit/e0b61b28641bdd20cfeff6d9878f1318b711ca19
|
|
Date |
User |
Action |
Args |
2022-04-11 14:59:41 | admin | set | github: 87385 |
2021-10-04 19:18:42 | pablogsal | set | nosy:
+ pablogsal messages:
+ msg403155
|
2021-09-21 22:21:18 | lukasz.langa | set | messages:
+ msg402394 |
2021-09-21 22:14:48 | lukasz.langa | set | messages:
+ msg402386 |
2021-09-21 21:53:47 | miss-islington | set | pull_requests:
+ pull_request26905 |
2021-09-21 21:53:43 | miss-islington | set | pull_requests:
+ pull_request26903 |
2021-09-21 21:53:11 | lukasz.langa | set | nosy:
+ lukasz.langa messages:
+ msg402373
|
2021-09-17 09:28:37 | Alex Grund | set | nosy:
+ Alex Grund messages:
+ msg402022
|
2021-07-20 19:03:44 | andrei.avk | set | messages:
+ msg397901 |
2021-07-20 18:53:47 | miss-islington | set | messages:
+ msg397898 |
2021-07-20 18:45:11 | miss-islington | set | messages:
+ msg397897 |
2021-07-20 18:16:59 | miss-islington | set | pull_requests:
+ pull_request25812 |
2021-07-20 18:16:35 | miss-islington | set | pull_requests:
+ pull_request25811 |
2021-07-20 18:16:28 | gregory.p.smith | set | messages:
+ msg397893 |
2021-07-20 10:18:58 | kulikjak | set | pull_requests:
+ pull_request25802 |
2021-07-20 10:17:56 | kulikjak | set | nosy:
+ kulikjak messages:
+ msg397865
|
2021-07-10 15:03:04 | gregory.p.smith | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
2021-07-10 04:14:07 | miss-islington | set | messages:
+ msg397237 |
2021-07-10 04:07:41 | miss-islington | set | messages:
+ msg397236 |
2021-07-10 03:48:03 | miss-islington | set | pull_requests:
+ pull_request25631 |
2021-07-10 03:47:59 | gregory.p.smith | set | messages:
+ msg397235 |
2021-07-10 03:47:58 | miss-islington | set | nosy:
+ miss-islington pull_requests:
+ pull_request25630
|
2021-07-09 05:04:39 | gregory.p.smith | set | assignee: gregory.p.smith
nosy:
+ gregory.p.smith versions:
+ Python 3.11, - Python 3.8 |
2021-07-07 21:18:58 | andrei.avk | set | messages:
+ msg397119 |
2021-07-07 03:26:04 | andrei.avk | set | messages:
+ msg397061 |
2021-07-07 03:23:12 | andrei.avk | set | messages:
+ msg397060 |
2021-07-07 02:29:32 | andrei.avk | set | keywords:
+ patch nosy:
+ andrei.avk
pull_requests:
+ pull_request25605 stage: patch review |
2021-03-17 07:30:11 | eryksun | link | issue35216 superseder |
2021-02-24 18:35:02 | eryksun | link | issue24977 superseder |
2021-02-17 02:02:47 | eryksun | set | type: behavior messages:
+ msg387134 components:
+ Library (Lib) versions:
+ Python 3.8, Python 3.9, Python 3.10, - Python 3.7 |
2021-02-14 06:49:04 | eryksun | set | nosy:
+ eryksun messages:
+ msg386935
|
2021-02-14 05:28:45 | jerpint | set | messages:
+ msg386933 |
2021-02-14 05:16:28 | jerpint | create | |