classification
Title: Shutil cannot delete a folder that contains an .ini file
Type: crash Stage: resolved
Components: IO, Windows Versions: Python 3.8
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: QueenSvetlana, eryksun, kinow, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2019-11-20 20:04 by QueenSvetlana, last changed 2019-12-06 16:43 by QueenSvetlana. This issue is now closed.

Messages (7)
msg357097 - (view) Author: Svetlana Vodianova (QueenSvetlana) Date: 2019-11-20 20:04
I posted my problem on Stack Overflow, but as of right now haven't received any answers. 

Link: https://stackoverflow.com/questions/58922332/shutil-cannot-delete-a-folder-with-a-hidden-desktop-ini-file

To summarize my problem: If you use copytree to copy a source folder that contains a desktop.ini file to a destination, and then call rmtree() to remove the source folder, it will throw an error indicating it doesn't have permission to remove the folder. Using ignore_patterns() doesn't help either, or to quote my problem on SO:

It seems once the source contains a .ini file, even if it isn't copied to the destination, rmtree() will have a problem removing the source folder.

Powershell has almost the same issue, however, unlike Python, if you tell PS to ignore the .ini file, it will remove the folder without crashing.

rmtree has no problem removing folders containing 0 ini files.
msg357099 - (view) Author: Svetlana Vodianova (QueenSvetlana) Date: 2019-11-20 20:10
I posted my problem on Stack Overflow, but as of right now haven't received any answers. 

Link: https://stackoverflow.com/questions/58922332/shutil-cannot-delete-a-folder-with-a-hidden-desktop-ini-file

To summarize my problem: If you use copytree to copy a source folder that contains a desktop.ini file to a destination, and then call rmtree() to remove the destination folder (for example after zipping, you may want to remove the uncompressed destination folder), it will throw an error indicating it doesn't have permission to remove the folder. Using ignore_patterns() doesn't help either, or to quote my problem on SO:

It seems once the source contains a .ini file, even if it isn't copied to the destination, rmtree() will have a problem removing the destination folder.

Powershell has almost the same issue, however, unlike Python, if you tell PS to ignore the .ini file, it will remove the folder without crashing.

rmtree has no problem removing folders containing 0 ini files.
msg357110 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-11-20 22:52
What is the system error code (winerror) of the PermissionError?
msg357171 - (view) Author: Svetlana Vodianova (QueenSvetlana) Date: 2019-11-21 14:31
What is the system error code (winerror) of the PermissionError?

PermissionError: [WinError 5] Access is denied


Note: Please ignore my first post (msg357097) describing the problem, I made a few mistakes that might be confusing. msg357099 better describes my problem.
msg357315 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-11-22 19:19
The directory probably has the "readonly" attribute set. The documentation includes a basic remove_readonly error handler for rmtree [1]:

    import os, stat
    import shutil
    
    def remove_readonly(func, path, _):
        "Clear the readonly bit and reattempt the removal"
        os.chmod(path, stat.S_IWRITE)
        func(path)

    shutil.rmtree(directory, onerror=remove_readonly)

That the readonly attribute prevents unlinking a directory is not particularly useful because a directory has to be empty anyway in order to unlink it. This is in contrast to the "immutable" file attribute in Linux, which prevents any modification of a directory, such as linking or unlinking files in it. Windows inherits its comparatively simplistic behavior for readonly directories from MS-DOS.

Since setting a directory as readonly is practically useless, the Windows shell reuses this attribute to indicate a customized folder [2]. If a directory is flagged readonly, the shell checks for and reads a "desktop.ini" file in it that can set a custom name, icon, tooltip, etc. Given you're copying a folder that contains a desktop.ini file, it's very likely that the readonly attribute is set. 

shutil.copytree will copy the readonly attribute to the destination directory via os.chmod. This is the only file attribute that gets copied (e.g. hidden and system aren't handled), due to a hack in the Windows CRT that conflates the readonly file attribute as a write 'permission' for chmod() and stat(). (Being readonly is an attribute of the file or directory, not a granted permission. os.access is the correct function to take both attributes and permissions into account, not os.stat.) Python inherits the CRT behavior in its own implementation of os.chmod and os.stat in Windows. It does so as an accident of history, not because it's justified.

[1] https://docs.python.org/3/library/shutil.html#rmtree-example
[2] https://docs.microsoft.com/en-us/windows/win32/shell/how-to-customize-folders-with-desktop-ini
msg357699 - (view) Author: Bruno P. Kinoshita (kinow) * Date: 2019-12-02 01:53
I think eryksun is correct. Reproduced it locally. Setting to read-only initially did not raise the issue, but then as administrator I removed the inherited permissions and set it to read-only for my user, then it raised the exact same WinError 5 Permission Error.

With the extra code to remove the readonly attribute, the directory was successfully deleted.

>It does so as an accident of history, not because it's justified

I think it would be easier for users if this attribute was not copied, especially as it looks like other attributes are not being copied (on Win). That would remove the need of this extra code when using Windows.
msg357923 - (view) Author: Svetlana Vodianova (QueenSvetlana) Date: 2019-12-06 16:43
I think eryksun is correct.

I agree. The code sample provided by him, and the Python docs, mean I'm able to remove a folder with a .ini file without getting WinError 5.

I should mark this as closed as he solved my problem.
History
Date User Action Args
2019-12-06 16:43:25QueenSvetlanasetstatus: open -> closed

messages: + msg357923
stage: resolved
2019-12-02 01:53:11kinowsetnosy: + kinow
messages: + msg357699
2019-11-22 19:19:56eryksunsetmessages: + msg357315
2019-11-21 14:31:26QueenSvetlanasetmessages: + msg357171
2019-11-20 22:52:26eryksunsetnosy: + eryksun
messages: + msg357110
2019-11-20 20:10:31QueenSvetlanasetmessages: + msg357099
2019-11-20 20:04:43QueenSvetlanacreate