Python 3.9.6 scan_dir returns filenotfound on long paths, but os_walk does not.  I've enclosed sample scripts that compare the two and have returned the results.  the windows 10 registry entry to extend the path names fixes this issue (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled set to 1).  I've enclosed a scripts that proved this occurs and can be used for testing. I have a script that does the same thing using os_walk, but I can't attach two scripts to this Issue.

Here's the second file that works just fine under python 3.9 (by the way, I am using Windows 64-bit).  I didn't test this on later python versions, however, nor did I test it on 32-bit versions.  I see that many people on the internet have said to change the working directory as a work around.  Could this possibly be why?

> Python 3.9.6 scan_dir returns filenotfound on long paths, 
> but os_walk does not.

This would be surprising. os.walk() has been implemented via os.scandir() since Python 3.5. Do you have a concrete example of the directory structure to test? 

> I see that many people on the internet have said to 
> change the working directory as a work around.  

Changing the working directory is a workaround in Unix, not Windows. Without long-path support, the working directory in Windows is limited to 258 (MAX_PATH - 2) characters.

Without long-path support, the workaround in Windows is to use an extended path, i.e. a Unicode path that's fully-qualified and normalized -- as returned by os.path.abspath() -- and prefixed by "\\\\?\\" or "\\\\?\\UNC\\" (e.g. r"\\?\C:\spam" or r"\\?\UNC\server\share\spam"). This allows the native path length limit of about 32760 characters. Some API functions and applications do not support extended paths. In particular setting an extended path as the working directory is unsupported and buggy, even if long-path support is enabled. But extended paths work fine with most file functions in the os and shutil modules, such as os.scandir(), os.stat(), os.open(), shutil.copytree(), and shutil.rmtree().

yes, I do.  C:\Users\Jim\Documents\jschw_uiowtv3_old\AppData\Local\Google\Chrome\User Data\Default\Extensions\nenlahapcbofgnanklpelkaejcehkggg\0.1.823.675_0\notifications\pages\Cashback\components\CashBackResolve\components\RewardsActivation\components\CashbackSectionSimple

it's over the 260 character limit that's the default for windows 10.

It works as expected for me:

    >>> len(p)
    261
    >>> print(p)
    C:\Temp\Jim\Documents\jschw_uiowtv3_old\AppData\Local\Google\Chrome\User Data\Default\Extensions\nenlahapcbofgnanklpelkaejcehkggg\0.1.823.675_0\notifications\pages\Cashback\components\CashBackResolve\components\RewardsActivation\components\CashbackSectionSimple

os.walk() can list the files in directory `p` if the \\?\ prefix is prepended:

    >>> next(os.walk('\\\\?\\' + p))[-1]
    ['spam.txt']

Without the prefix, the internal os.scandir() call fails, but by default the error is ignored:

    >>> next(os.walk(p))[-1]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    StopIteration

We can print the exception to see that it's the expected ERROR_PATH_NOT_FOUND (3) error for a path that's too long:

    >>> next(os.walk(p, onerror=print))[-1]
    [WinError 3] The system cannot find the path specified: 'C:\\Temp\\Jim\\Documents\\jschw_uiowtv3_old\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Extensions\\nenlahapcbofgnanklpelkaejcehkggg\\0.1.823.675_0\\notifications\\pages\\Cashback\\components\\CashBackResolve\\components\\RewardsActivation\\components\\CashbackSectionSimple'
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    StopIteration

do you have this registry entry set to 1: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled set to 1.  It works if you do.  What version of windows do you have?  I have version 21H2 (OS Build 19044.1387).  I don't have windows 11 yet.

If I had long paths enabled, then next(os.walk(p, onerror=print)) would not have printed the error that I showed in the example and would not have immediately raised StopIteration. Instead it would have returned a (dirpath, dirnames, filenames) result for directory `p`. Did you repeat the simple examples that I showed, exactly as shown, and get a different result?

os.walk() has been implemented via os.scandir(), but by default it ignores OSErrors raised by os.scandir(), DirEntry.is_dir() and DirEntry.is_symlink(). You can get errors raised by os.scandir() if specify the onerror argument, but errors in DirEntry.is_dir() and DirEntry.is_symlink() are always ignored, so too deep directories or links to directories can be treated as files.

> but errors in DirEntry.is_dir() and DirEntry.is_symlink() 
> are always ignored

In Windows, is_symlink() won't fail due to a long path, since that information comes from the directory listing, but is_dir() might fail for a long path if it's a symlink to a directory. Windows requires that a symlink to a directory is also a directory (i.e. the symlink reparse point is set on an empty directory), but it's not enough to check that it's a directory symlink. is_dir() requires checking that the target exists, which may fail if the path of the link is too long to open and resolve the link target.

the issue is with the scandir script, not the os_walk script.  I tried to upload the scandir python script before, but I guess it didn't upload.  When I was running the two scripts, I used an input of C:\\ as the input parameter.  Hope that helps.

when I run the following command:

python "H:\Users\LindaJim\Documents\AWS Python Learning\test_dir_scan_dir.py" "C:\\"

I get this output:

...
Traceback (most recent call last):
  File "H:\Users\LindaJim\Documents\AWS Python Learning\test_dir_scan_dir.py", line 54, in <module>
    main(sys.argv[0:])
  File "H:\Users\LindaJim\Documents\AWS Python Learning\test_dir_scan_dir.py", line 30, in main
    for file in get_files_in_dir(source):
  File "H:\Users\LindaJim\Documents\AWS Python Learning\test_dir_scan_dir.py", line 11, in get_files_in_dir
    yield from get_files_in_dir(entry.path)
  File "H:\Users\LindaJim\Documents\AWS Python Learning\test_dir_scan_dir.py", line 11, in get_files_in_dir
    yield from get_files_in_dir(entry.path)
  File "H:\Users\LindaJim\Documents\AWS Python Learning\test_dir_scan_dir.py", line 11, in get_files_in_dir
    yield from get_files_in_dir(entry.path)
  [Previous line repeated 19 more times]
  File "H:\Users\LindaJim\Documents\AWS Python Learning\test_dir_scan_dir.py", line 9, in get_files_in_dir
    for entry in os.scandir(source):
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Jim\\Documents\\jschw_uiowtv3_old\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Extensions\\nenlahapcbofgnanklpelkaejcehkggg\\0.1.823.675_0\\notifications\\pages\\Cashback\\components\\CashBackResolve\\components\\RewardsActivation\\components\\CashbackSectionSimple'

when I run the following command:

python "H:\Users\LindaJim\Documents\AWS Python Learning\test_os_walk.py" "C:\\"

I get this:

...
file is  C:\winutils\bin\winutils.exe
End time is  2021-12-15.13:11:54
Duration is  0:06:05

I don't think this should happen, right?

my c drive and h drive are both internal drives and I run the python script from my user directory on my c drive.  Not sure if that makes any difference.  Just trying to think of things that might help you reproduce and fix this.

Please let me know if you are able to reproduce this issue.