classification
Title: Bad error message in os.rename, os.link, and os.symlink
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder: Support errors with two filenames for errno exceptions
View: 20517
Assigned To: larry Nosy List: Arfrever, ezio.melotti, georg.brandl, jderose, jwilk, kushal.das, larry, serhiy.storchaka, terry.reedy, v+python, vajrasky
Priority: normal Keywords: 3.3regression, patch

Created on 2012-09-28 08:13 by v+python, last changed 2014-02-24 20:45 by larry. This issue is now closed.

Files
File name Uploaded Description Edit
test93.py v+python, 2012-09-28 08:13
larry.rename_error.1.diff larry, 2012-09-28 20:13 First draft at a patch. review
larry.rename_error.2.diff larry, 2012-09-28 20:45 review
better_error_message_in_os_rename_link_symlink.patch vajrasky, 2013-12-02 04:11 For Python 3.4 (removed file information) review
better_error_message_in_os_rename_link_symlink_for_python_33.patch vajrasky, 2013-12-02 04:33 For Python 3.3 (removed file information) review
Messages (28)
msg171410 - (view) Author: Glenn Linderman (v+python) * Date: 2012-09-28 08:13
I've been using 3.3.0b1 for development, with mostly no problems, but today I was surprised and confused by an error message. It is an attempt to be an improvement over 3.2, giving the filename that os.rename cannot find... but instead, it gives the one it is renaming to.

So I quickly upgraded to 3.3.0rc3, and it contains the same error.
msg171515 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-09-28 18:06
Could you give a specific example of the message you got, why it is wrong, and what you think it should be?
Is there any reason to think that this is Windows-specific?
msg171517 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-09-28 18:32
I confirm the bug.

>>> import os
>>> os.rename('non-existent-name', 'new-name')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'new-name'

It obviously should be "No such file or directory: 'non-existent-name'".
msg171524 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-09-28 18:58
Error in argument of path_error in function internal_rename.

For some errcode it should be dst, for some src, for some both, for some none, and for some it is unknown. And this can be platform-specific.

I suggest rollback this enhancement.
msg171525 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-09-28 18:59
> Error in argument of path_error in function internal_rename.

In file Modules/posixmodule.c.
msg171528 - (view) Author: Glenn Linderman (v+python) * Date: 2012-09-28 19:13
Terry, I thought the test case would demonstrate the error details.
As far as Windows, I encountered it there, and it seemed like the sort of error that could be in a Windows-specific module.

Serhiy, thanks for confirming, and analyzing. At this point in the release cycle, rolling back the enhanced message is probably better than trying to fix it.  The 3.2 message was imprecise, but not confused with the sometimes-wrong file name.
msg171538 - (view) Author: Kushal Das (kushal.das) * (Python committer) Date: 2012-09-28 20:10
Now the printing the source name instead of the destination as the bug described.
msg171539 - (view) Author: Kushal Das (kushal.das) * (Python committer) Date: 2012-09-28 20:12
Just for record the same issue can be seen in a Linux box also. I tested in a Fedora 17 system.
msg171540 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2012-09-28 20:13
As Serhiy says, which filename to display is wholly context-sensitive to which errno it is.  And there's no cheap way to display both.  The cheap fix is to call path_error with a nulled-out "path" object; this will display the error without a filename.

Patch doing this attached.

By default we can't fix this in 3.3.0.  I'll add Georg on the off chance he wants to cherry-pick it.

Perhaps for 3.3.1 I can fix this properly, with a switch on the errno.
msg171543 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2012-09-28 20:45
New patch, just calling posix_error directly.
msg171583 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-09-29 18:10
Same issue for os.link and os.symlink.

>>> import os
>>> os.link('non-existent-name', 'new-name')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'new-name'
>>> os.symlink('long-name'*1000, 'new-name')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 36] File name too long: 'new-name'
msg178513 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-29 16:57
As I see, now 3.4 behavior differs from 3.3 behavior.

>>> os.link('non-existent-name', 'new-name')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'non-existent-name'
>>> os.symlink('long-name'*1000, 'new-name')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 36] File name too long: 'long-namelong-namelong-namelong-namelong-namelong-name...
msg186756 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-04-13 16:45
I just closed two almost identical issues in favour of this issue. I believe that in order to avoid ambiguity, there are three possible solutions of such issues:

1. Do not display any filename (as in Python 2.7).
2. Display both filenames (this can also be slightly ambiguous).
3. Depending on the errno display the first filename or the second filename, or neither, or both.
msg186772 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-04-13 17:33
I would say either option 3, or, if it's too complicated, option 2.
FTR the two issues mentioned in the previous message are #13775 and #16812.
msg204991 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2013-12-02 04:11
Here is the patch for Python 3.4. It removed file information for os.rename, os.link, and os.symlink.

I agree with Ezio that giving extra information is better but since Python 3.4 is in beta.... maybe we can defer this to Python 3.5.

There are couples of errno that need extra information. For example, in Linux:
[Errno 2] No such file or directory -> source
[Errno 17] File exists -> destination
[Errno 36] Filename too long -> tough to decide, can be source, can be destination
[Errno 27] File too large -> source
Anything left?
msg204992 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2013-12-02 04:33
Patch for Python 3.3. It omitted file information for error messages in rename, symlink, and link.

I'll create a separate ticket for adding extra file information to these error messages.
msg206589 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2013-12-19 08:45
Is there any possibility this ticket could be committed in Python 3.4? If yes, it would be good because we would have a good foundation for creating better error message in Python 3.5.

Anyway, I check Python's competitors' behaviour.

Ruby displays both files.

irb(main):001:0> File.symlink('a.txt', 'b')
Errno::EEXIST: File exists @ sys_fail2 - (a.txt, b)
	from (irb):1:in `symlink'
	from (irb):1
	from /home/ethan/Documents/code/ruby/localruby/bin/irb:11:in `<main>'

Perl omits the files.

  DB<8> print symlink("a.txt", "b");
0
  DB<9> print $!;
File exists

PHP omits the files.

php -r "symlink('b', 'a');"
PHP Warning:  symlink(): File exists in Command line code on line 1
PHP Stack trace:
PHP   1. {main}() Command line code:0
PHP   2. symlink() Command line code:1

Warning: symlink(): File exists in Command line code on line 1

Call Stack:
    0.0001     225688   1. {main}() Command line code:0
    0.0001     226392   2. symlink() Command line code:1
msg206765 - (view) Author: Jakub Wilk (jwilk) Date: 2013-12-21 16:57
As far as rename() and link() are concerned, either of the arguments can cause an ENOENT error:

os.rename('/dev/foobar', '/dev/barfoo') # fails because /dev/foobar doesn't exist

os.rename('/dev/null', '/foo/bar/baz') # fails because /foo/bar doesn't exist
msg207059 - (view) Author: Jason Gerard DeRose (jderose) Date: 2013-12-29 01:11
vajrasky, one more errno to consider:

[Errno 39] Directory not empty: '/tmp/bar'

This is when renaming a directory: src directory exists; dst directory exists and is non-empty.

In 3.4 Beta1, this error message now includes the src instead of the dst (flipped behaviour from Python 3.3). I'm not sure whether this changed because of changes being tracked in this bug on not, so I filed a separate issue: http://bugs.python.org/issue20093
msg207420 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2014-01-06 10:24
Jakub Wilk, yeah, I missed that one.

Jason Gerard DeRose, thanks for the additional information.

Anyway, I already "persuaded" Ruby developers to specialize the message, although it's only for creating symlink.

https://bugs.ruby-lang.org/issues/9263

As of Python, let's leave that for Python 3.5.
msg209327 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-26 15:05
The 3.4 patch LGTM, but the 3.3 patch perhaps has a bug.
msg210016 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-02 20:51
Larry, are you agree with this solution (remove ambiguous filename attribute from OSError raised by functions with two filename arguments)?
msg210129 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-02-03 14:24
As release manager, I would be willing to consider this patch.  However, as original author of the code in question, I don't like it.  Showing zero filenames, while potentially less confusing, is also far less helpful.  A better solution would be to show both filenames.  This would require changing the OSError exception, and adding some new PyErr_ API calls.  I'm now working on a patch to do that.
msg210165 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-03 20:38
However it is still a bug in 3.3.
msg210363 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-02-06 07:54
I probably should have continued with this issue instead of creating a new issue for it, sorry.  But there's a new issue tracking this change, where I propose to add new error-reporting functions that support two filenames.  That's happening in #20517.
msg210800 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2014-02-10 06:48
Now, that we have fixed this bug in Python 3.4 with this commit http://hg.python.org/cpython/rev/081a9d8ba3c7, what should we do with the bug in Python 3.3?

Use my patch (omitting filenames)?

Keep the status quo (one filename)? If yes, close this ticket as invalid.

Backport Larry's commit (two filenames) which is not straightforward?
msg212135 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-02-24 19:39
No change for 3.3.
msg212138 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-02-24 20:45
It's fine that we're not fixing this in 3.3, but since this did get fixed in 3.4, I propose dropping 3.3 from the version list and changing the resolution to "fixed".

*waves magic wand*
History
Date User Action Args
2014-02-24 20:45:38larrysetresolution: wont fix -> fixed
stage: needs patch -> resolved
messages: + msg212138
versions: - Python 3.3
2014-02-24 19:39:52georg.brandlsetstatus: open -> closed
resolution: wont fix
messages: + msg212135
2014-02-10 06:48:41vajraskysetmessages: + msg210800
2014-02-06 07:54:43larrysetsuperseder: Support errors with two filenames for errno exceptions
messages: + msg210363
2014-02-03 20:38:27serhiy.storchakasetassignee: serhiy.storchaka -> larry
messages: + msg210165
2014-02-03 14:24:14larrysetmessages: + msg210129
2014-02-02 20:51:21serhiy.storchakasetmessages: + msg210016
2014-01-26 15:05:31serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg209327
2014-01-06 10:24:09vajraskysetmessages: + msg207420
2013-12-29 01:11:10jderosesetnosy: + jderose
messages: + msg207059
2013-12-21 16:57:04jwilksetnosy: + jwilk
messages: + msg206765
2013-12-19 08:45:29vajraskysetmessages: + msg206589
2013-12-02 04:33:47vajraskysetfiles: + better_error_message_in_os_rename_link_symlink_for_python_33.patch

messages: + msg204992
2013-12-02 04:11:55vajraskysetfiles: + better_error_message_in_os_rename_link_symlink.patch
nosy: + vajrasky
messages: + msg204991

2013-12-02 03:23:35vajraskysettitle: bad error message in os.rename -> Bad error message in os.rename, os.link, and os.symlink
2013-12-01 16:09:43serhiy.storchakasetkeywords: + 3.3regression
2013-12-01 16:08:13serhiy.storchakalinkissue19848 superseder
2013-07-10 01:40:21ned.deilylinkissue18420 superseder
2013-04-13 17:33:05ezio.melottisetmessages: + msg186772
2013-04-13 16:45:53serhiy.storchakasetmessages: + msg186756
versions: + Python 3.4
2013-04-13 16:30:06serhiy.storchakalinkissue13775 superseder
2013-04-13 16:29:32serhiy.storchakalinkissue16812 superseder
2012-12-29 16:57:06serhiy.storchakasetmessages: + msg178513
2012-09-30 22:36:19Arfreversetnosy: + Arfrever
2012-09-30 17:03:44ezio.melottisetnosy: + ezio.melotti
2012-09-29 18:10:53serhiy.storchakasetmessages: + msg171583
2012-09-28 20:45:55larrysetfiles: + larry.rename_error.2.diff

messages: + msg171543
2012-09-28 20:13:51kushal.dassetfiles: - os_rename.patch
2012-09-28 20:13:32larrysetfiles: + larry.rename_error.1.diff
nosy: + georg.brandl
messages: + msg171540

2012-09-28 20:12:23kushal.dassetmessages: + msg171539
2012-09-28 20:10:46kushal.dassetfiles: + os_rename.patch

nosy: + kushal.das
messages: + msg171538

keywords: + patch
2012-09-28 19:13:04v+pythonsetmessages: + msg171528
2012-09-28 18:59:14serhiy.storchakasetmessages: + msg171525
2012-09-28 18:58:07serhiy.storchakasetnosy: + larry
messages: + msg171524
2012-09-28 18:32:58serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg171517
2012-09-28 18:06:32terry.reedysettype: behavior
components: + Library (Lib), - Windows
title: bad error in rename -> bad error message in os.rename
nosy: + terry.reedy

messages: + msg171515
stage: needs patch
2012-09-28 08:13:18v+pythoncreate