Author Samson Lee
Recipients Samson Lee
Date 2015-12-14.10:30:54
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1450089055.62.0.989850865471.issue25860@psf.upfronthosting.co.za>
In-reply-to
Content
I noticed os.fwalk() produced different results as os.walk(). It turned out
that os.fwalk() skips all remaining directories when OSError occurs (e.g. due
to PermissionError).

To reproduce the bug, first create a test directory structure:

    $ mkdir 1; touch 1/a
    $ mkdir 2; touch 2/b
    $ mkdir 3; touch 3/c
    $ mkdir 4; touch 4/c

At this stage, everything is okay, both os.fwalk() os.walk() produce the same
results:

    >>> import os
    >>> for root, dirs, files in os.walk('.'):
    ...     dirs.sort()
    ...     print(root, dirs, files)
    . ['1', '2', '3', '4'] []
    ./1 [] ['a']
    ./2 [] ['b']
    ./3 [] ['c']
    ./4 [] ['d']
    >>> for root, dirs, files, fd in os.fwalk('.'):
    ...     dirs.sort()
    ...     print(root, dirs, files)
    . ['1', '2', '3', '4'] []
    ./1 [] ['a']
    ./2 [] ['b']
    ./3 [] ['c']
    ./4 [] ['d']

To introduce an error, force a PermissionError on one of the directories:

    $ sudo chown root:root 2
    $ sudo chmod 700 2

Now, the os.fwalk() results are different (trust me, os.walk() is still okay):

    >>> for root, dirs, files, fd in os.fwalk('.'):
    ...     dirs.sort()
    ...     print(root, dirs, files)
    . ['1', '2', '3', '4'] []
    ./1 [] ['a']

So it seems that os.fwalk skips remaining directories after an error occurs.

The cause of the problem is in this part of os.py:

    for name in dirs:
        try:
            orig_st = stat(name, dir_fd=topfd, follow_symlinks=follow_symlinks)
            dirfd = open(name, O_RDONLY, dir_fd=topfd)
        except OSError as err:
            if onerror is not None:
                onerror(err)
            break

To fix it, simply replace break with continue. Patch attached.

Cheers
History
Date User Action Args
2015-12-14 10:30:55Samson Leesetrecipients: + Samson Lee
2015-12-14 10:30:55Samson Leesetmessageid: <1450089055.62.0.989850865471.issue25860@psf.upfronthosting.co.za>
2015-12-14 10:30:55Samson Leelinkissue25860 messages
2015-12-14 10:30:54Samson Leecreate