classification
Title: shutils test fails on ZFS (on FUSE, on Linux)
Type: Stage:
Components: Build Versions: Python 3.1, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, larry
Priority: normal Keywords: patch

Created on 2009-04-03 00:46 by larry, last changed 2009-04-29 22:44 by benjamin.peterson. This issue is now closed.

Files
File name Uploaded Description Edit
lch.make.clean.diff larry, 2009-04-03 00:46 Patch against py3k/trunk r71076.
lch.test_shutil.r71304.diff larry, 2009-04-06 19:25
lch.test_shutil.r71304.v2.diff larry, 2009-04-06 19:26
lch.test_shutil.r71404.diff larry, 2009-04-09 08:36
lch.test_shutil.r71641.diff larry, 2009-04-16 09:36 Patch against py3k/trunk svn r71641.
Messages (12)
msg85285 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2009-04-03 00:46
"make clean" fails in my tree.  There's an inscrutable new directory,
"@test", which is chmod 400, and therefore "find" isn't allowed to
descend into it, so it fails and aborts the clean early.  I don't know
what the deal is here, but I patched the rules for "clean" and
"pycremoval" so they chmod "@test" temporarily to 755 before running and
back to "400" afterwards.

A better fix: get rid of this "chmod 400 @test" nonsense.  But I'm too
lazy to investigate why it was done in the first place.  Or tell me what
I'm doing wrong in the first place...
msg85288 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-04-03 01:38
@test is the file used by tests. It shouldn't be left around by tests,
so if it is, I think you should explicitly has to remove it.
msg85621 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2009-04-06 08:45
So what's the fix?  I upgraded to r71304 and it still does this.  Worse,
it adds all sorts of complaints to "make test"--and it actually slows
down hg.  (hg can't cd into that directory, so it gives up on creating
its inotify socket.)  Can whoever added "@test" fix it please--or tell
me what I could possibly be doing wrong?  Thanks.
msg85644 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-04-06 13:34
2009/4/6 Larry Hastings <report@bugs.python.org>:
>
> Larry Hastings <larry@hastings.org> added the comment:
>
> So what's the fix?  I upgraded to r71304 and it still does this.  Worse,
> it adds all sorts of complaints to "make test"--and it actually slows
> down hg.  (hg can't cd into that directory, so it gives up on creating
> its inotify socket.)  Can whoever added "@test" fix it please--or tell
> me what I could possibly be doing wrong?  Thanks.

$ rm -rf @test
msg85664 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2009-04-06 19:25
% rm -rf @test
rm: cannot remove `@test': Permission denied

And, before you helpfully chime in with "chmod", yes I know how to run
that too.

I stuck in and figured it out.  The bug: "test_on_error" in
"test_shutil" is out of sync with how "shutil.rmtree" runs. 
"test_on_error" deliberately constructs a directory that is not
executable to the current user, then calls "shutil.rmtree" to delete it,
passing in an "onerror" callback of "check_args_to_onerror" so it can
observe how it fails.  "check_args_to_onerror". expects to be called
twice; the first time it expects the failing function to be "os.remove".
 In fact, the first failure is "os.listdir".  This fails, throws an
exception, and the test is over.  Which means the cleanup code in
"test_on_error" (the subsequent "chmod" and "rmtree" calls) don't get
executed, the "@test" with mode 400 is left behind, and now we have
cascading errors.

Attached is a patch that fixes "check_args_to_onerror" on my machine. 
For good measure, it also changes "cleanup_test_droppings" in "regrtest"
cleanup code so it chmods a directory to 0o700 before attempting to
remove it.  The patch was written against py3k/trunk r71304.

I find it hard to believe that I'm the only person running "make test"
in py3k/trunk not on Windows and not as root.  So I theorize os.listdir
behaves differently on different OSes.  I'm on Ubuntu 9.04, and
os.listdir fails if called on a directory that is not executable. 
Perhaps, on the omnipresent Macs, this is permissible?  If so, the tests
in "check_args_on_error" when errorState is 0 should be rewritten,
either to accept any of the possible arguments or to branch on
sys.platform and accept specific arguments.
msg85665 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2009-04-06 19:26
Crapdoodles, I didn't update the patch to get the "regrtest" change I
threw in at the last minute.  Please see the "v2" patch.
msg85798 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2009-04-09 08:36
The analysis I wrote on 2009/4/6 is correct, but I just discovered
another important detail: it only happens on my ZFS partition.  I'm
using ZFS on Linux, via FUSE.  No wonder I'm the only person it's
happening to!  I don't know whether it's because of FUSE or because of
ZFS; if it's because of ZFS, then more people will see this once the
next rev of OS X hits the streets.

To reproduce:
% mkdir foo
% touch foo/a
% chmod 400 foo
% python -c 'import os ; print(os.listdir("foo"))'

On my ext4 root partition this works, printing
  ['a']
On my ZFS home partition this fails, printing
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  OSError: [Errno 13] Permission denied: 'foo'

I observe this behavior in Python 2.6.1 and in py3k/trunk.

The attached patch fixes test_shutil so it passes on both partitions. 
The fix: allow the first failure to be *either* os.listdir (in which
case arg must be TESTFN) or os.remove (in which case arg must be
self.childpath).  Patch was written against r71404.
msg85799 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2009-04-09 08:38
I should be more specific: patch was written against py3k/trunk r71404.
 Though I imagine a very similar patch could be written for 2.7 (or 2.6
if appropriate).
msg85999 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-04-15 21:05
1. You should always compare functions with "is".
2. You should add a comment explaining why os.listdir is special cased.
msg86017 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2009-04-16 09:35
Done and done.  I also changed cleanup_test_droppings() in
Lib/test/regrtest.py; it now chmods the, uh, droppings before it
attempts their removal.
msg86823 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2009-04-29 20:29
Ping?  Is the fate of this patch waiting on anything I can help with?

In case it helps you decide, you don't see this behavior on a ZFS
partition on Mac OS X.  It appears to only happen under FUSE.  So it's
just me and my computers.  (And maybe Sean Reifschneider.)
msg86833 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-04-29 22:44
Fixed in r72131 and r72133.
History
Date User Action Args
2009-04-29 22:44:43benjamin.petersonsetstatus: open -> closed
resolution: fixed
messages: + msg86833
2009-04-29 20:29:00larrysetmessages: + msg86823
2009-04-16 09:36:26larrysetfiles: + lch.test_shutil.r71641.diff

messages: + msg86017
2009-04-15 21:05:06benjamin.petersonsetmessages: + msg85999
2009-04-09 08:38:12larrysetmessages: + msg85799
2009-04-09 08:36:59larrysetfiles: + lch.test_shutil.r71404.diff

title: Fix "make clean" in py3k/trunk -> shutils test fails on ZFS (on FUSE, on Linux)
messages: + msg85798
versions: + Python 2.6
2009-04-06 19:26:33larrysetfiles: + lch.test_shutil.r71304.v2.diff

messages: + msg85665
2009-04-06 19:25:18larrysetfiles: + lch.test_shutil.r71304.diff

messages: + msg85664
2009-04-06 13:34:47benjamin.petersonsetmessages: + msg85644
2009-04-06 08:45:08larrysetmessages: + msg85621
2009-04-03 01:38:00benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg85288
2009-04-03 00:46:55larrycreate