Author eryksun
Recipients eryksun, hroncok, jkloth, pablogsal, paul.moore, petr.viktorin, steve.dower, tim.golden, vstinner, zach.ware
Date 2019-10-15.02:54:52
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1571108093.22.0.601340157092.issue38470@roundup.psfhosted.org>
In-reply-to
Content
I tested in Windows 10 with long paths disabled, and the FileNotFoundError exception showed that the failing .pyc had a numeric suffix appended to the name.

create_long_path isn't taking into account _write_atomic in Lib/importlib/_bootstrap_external.py. This function appends the id() of the target path string in order to create a temporary unique name, which it renames to the target if the write succeeds. Since the object ID is a memory address, create_long_path would need to append 16 characters to the longer_cache path. That's 1 for the initial "." of the suffix, plus 15 for the x64 address-space limit of 128 TB, i.e. len(str(0x7FFF_FFFF_FFFF)). 

Note that, by default, long-path support is not enabled in Windows 10. At process startup, the system runtime library enables long-path support if "LongPathsEnabled" is set in "HKLM\SYSTEM\CurrentControlSet\Control\FileSystem" and the application manifest enables "longPathAware". Both settings are required.

For the buildbots, pythoninfo checks this as "windows.RtlAreLongPathsEnabled". According to this, for example, bolen-windows10 does not have long paths enabled. Yet test_compileall passed. 

bolen-windows10
https://buildbot.python.org/all/#/builders/3/builds/3681/steps/3/logs/stdio

It seems the test passed because the temp directory on this bot is relatively short: "D:\Temp". That's a factor because create_long_path adds directory components in blocks of 10. It uses 20 "dir_X_Y" components because 30 (240 characters) is too much for MAX_PATH no matter what. 

The same applies to the following buildbots for Windows 7 and 8.1:

bolen-windows7
https://buildbot.python.org/all/#/builders/58/builds/3148/steps/3/logs/stdio

ware-win81-release
https://buildbot.python.org/all/#/builders/12/builds/3360/steps/3/logs/stdio

On the other hand, kloth-win64 has a longer path for its temp directory: "C:\Users\Buildbot\AppData\Local\Temp". This costs an additional 29 characters compared to "D:\Temp" and brings the final path very close to MAX_PATH. The suffix appended by _write_atomic is typically 14-16 characters, which then exceeds MAX_PATH.
History
Date User Action Args
2019-10-15 02:54:53eryksunsetrecipients: + eryksun, paul.moore, vstinner, tim.golden, jkloth, petr.viktorin, zach.ware, steve.dower, hroncok, pablogsal
2019-10-15 02:54:53eryksunsetmessageid: <1571108093.22.0.601340157092.issue38470@roundup.psfhosted.org>
2019-10-15 02:54:53eryksunlinkissue38470 messages
2019-10-15 02:54:52eryksuncreate