This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eryksun
Recipients eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Date 2020-01-12.14:27:39
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1578839260.17.0.288692192504.issue39312@roundup.psfhosted.org>
In-reply-to
Content
Windows 10 apparently defaults to disguising placeholder reparse points in python.exe processes, but exposes them to cmd.exe and powershell.exe processes. 

A common example is a user's OneDrive folder, which extensively uses placeholder reparse points for files and directories. The placeholder file attributes include FILE_ATTRIBUTE_REPARSE_POINT, FILE_ATTRIBUTE_OFFLINE, and FILE_ATTRIBUTE_SPARSE_FILE, and the reparse tags are in the set
IO_REPARSE_TAG_CLOUD[_1-F] (0x9000[0-F]01A). Currently, we don't see any of this information in a python.exe process when we call FindFirstFile[Ex]W, GetFileAttributesW, or query file information on a file opened with FILE_FLAG_OPEN_REPARSE_POINT, such as when we call os.lstat. 

The behavior is determined by the process or per-thread placeholder-compatibility mode. The process mode can be queried via RtlQueryProcessPlaceholderCompatibilityMode [1]. The documentation says that "[m]ost Windows applications see exposed placeholders by default". I don't know what criteria Windows is using here, but in my tests with python.exe and a simple command-line test program, the default mode is PHCM_DISGUISE_PLACEHOLDER.

Should Python provide some way to call RtlSetProcessPlaceholderCompatibilityMode [2] to set PHCM_EXPOSE_PLACEHOLDERS mode for the current process? Should os.lstat be modified to temporarily expose placeholders -- for the current thread only -- via RtlSetThreadPlaceholderCompatibilityMode [3]? We can dynamically link to this ntdll function via GetProcAddress. It returns the previous mode, which we can restore after querying the file.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlqueryprocessplaceholdercompatibilitymode
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlsetprocessplaceholdercompatibilitymode
[3] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlsetthreadplaceholdercompatibilitymode
History
Date User Action Args
2020-01-12 14:27:40eryksunsetrecipients: + eryksun, paul.moore, tim.golden, zach.ware, steve.dower
2020-01-12 14:27:40eryksunsetmessageid: <1578839260.17.0.288692192504.issue39312@roundup.psfhosted.org>
2020-01-12 14:27:40eryksunlinkissue39312 messages
2020-01-12 14:27:39eryksuncreate