`pathlib.Path.iterdir()` uses `os.listdir()` rather than `os.scandir()`. I think this has a small performance cost, per PEP 471:

> It returns a generator instead of a list, so that scandir acts as a true iterator instead of returning the full list immediately.

As `scandir()` is already available from `_NormalAccessor` it's a simple patch to use `scandir()` instead.

This optimisation was also hinted at https://bugs.python.org/issue26032#msg257653

It is not so easy. There was reason why it was not done earlier. scandir() wastes more limited resource than memory -- file descriptors. It should also be properly closed and do not depend on the garbage collector.

Consider the example:

def traverse(path, visit):
    for child in path.iterdir():
        if child.is_dir():
            traverse(path, visit)
        else:
            visit(child)

With your optimization it may fail with OSError: Too many open files.

Ah, right you are! The globbing helpers call `list(os.scandir(...))` - perhaps we should do the same here?

It would be slower and less reliable implementation of os.listdir().

Less reliable how? Doesn't appear any slower:

barney.gale@heilbron:~$ python3 -m timeit -s "import os; os.listdir('/usr/local')"
100000000 loops, best of 3: 0.0108 usec per loop
barney.gale@heilbron:~$ python3 -m timeit -s "import os; list(os.scandir('/usr/local'))"
100000000 loops, best of 3: 0.00919 usec per loop

This is not how timeit works, you just measured the time taken by an empty loop, you can look at `python3 -m timeit -h` to get help how to call it. I think a correct invocation would be:

(venv) ➜  ~ python3 -m timeit -s 'from os import scandir' "list(scandir('/usr/local'))"
10000 loops, best of 5: 24.3 usec per loop
(venv) ➜  ~ python3 -m timeit -s 'from os import listdir' "listdir('/usr/local')"
10000 loops, best of 5: 22.2 usec per loop

so it looks like scandir as a small overhead when accumulating all results and not using the extra info it returns.

> Less reliable how?

See issue39916.

> so it looks like scandir as a small overhead when accumulating all results and not using the extra info it returns.

Try with larger directories. The difference may be not so small.

$ python3 -m timeit -s 'from os import scandir' "list(scandir('/usr/include'))"
10000 loops, best of 3: 176 usec per loop
$ python3 -m timeit -s 'from os import listdir' "listdir('/usr/include')"
10000 loops, best of 3: 114 usec per loop

Thanks Rémi and Serhiy! Closing this ticket as the patch doesn't provide any sort of improvement.