classification
Title: xml.ElementTree iterparse filehandle left open
Type: security Stage:
Components: XML Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: anthonypjshaw, dtom9424, vkisforever
Priority: normal Keywords:

Created on 2021-02-22 04:11 by vkisforever, last changed 2021-02-23 14:18 by dtom9424.

Files
File name Uploaded Description Edit
ETiterparse.py vkisforever, 2021-02-22 04:11 iterparse issue recreator
Messages (6)
msg387494 - (view) Author: Viktor Kis (vkisforever) * Date: 2021-02-22 04:11
Standard library xml.ElementTree - iterparse does not close filehandle properly when breaking out of a for loop iterating with iterparse. The issue only occurs when the user passes a "string" as a source object instead of a filehandle (that is because a user's filehandle can safely be closed in a context manager as shown in the attachment).
msg387495 - (view) Author: anthony shaw (anthonypjshaw) * (Python triager) Date: 2021-02-22 04:21
which version of Python were you doing this on?

The function in question is https://github.com/python/cpython/blob/master/Lib/xml/etree/ElementTree.py#L1233-L1278
msg387496 - (view) Author: Viktor Kis (vkisforever) * Date: 2021-02-22 04:22
ran python 3.9.2 on windows 10

1) What did you do?
Tried to remove folder that contains a xml file that was worked on by ElementTree iterparse with argument source='filename' (string type input)
2. What happened?
python was not able to remove the folder due to file lock still held by iterparse.
3. What did you expect to happen
iterparse was supposed to close the filehandle such that the user could then write/move/delete the xml file worked on, however python gave a PermissionError:

Traceback (most recent call last):
  File "C:\Users\vkisf\Desktop\00_MYSTUFF\01_SOFTWARE\01_PYTHON\01_GITHUB\pycascades2021\ETiterparse.py", line 43, in <module>
    shutil.rmtree('subdir2')
  File "C:\Users\vkisf\Desktop\00_MYSTUFF\01_SOFTWARE\01_PYTHON\01_GITHUB\pycascades2021\cpython\lib\shutil.py", line 740, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\vkisf\Desktop\00_MYSTUFF\01_SOFTWARE\01_PYTHON\01_GITHUB\pycascades2021\cpython\lib\shutil.py", line 618, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\vkisf\Desktop\00_MYSTUFF\01_SOFTWARE\01_PYTHON\01_GITHUB\pycascades2021\cpython\lib\shutil.py", line 616, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'subdir2\\app.xml'
msg387497 - (view) Author: anthony shaw (anthonypjshaw) * (Python triager) Date: 2021-02-22 04:32
Example script attached works perfectly on macOS 

> python3.9 test.py
[('default', 'http://schemas.openxmlformats.org/officeDocument/2006/extended-properties'), ('vt', 'http://schemas.openxmlformats.org/officeDocument/2006/docPropsVTypes')]
[('default', 'http://schemas.openxmlformats.org/officeDocument/2006/extended-properties'), ('vt', 'http://schemas.openxmlformats.org/officeDocument/2006/docPropsVTypes')]
msg387498 - (view) Author: Viktor Kis (vkisforever) * Date: 2021-02-22 04:37
Confirmed that no apps were using the created xml file or folder. Windows Explorer was closed for all folders only the terminal shell was open. Issue still persists on windows 10 OS.
msg387571 - (view) Author: Tom Dougherty (dtom9424) Date: 2021-02-23 14:18
"erase all files"
History
Date User Action Args
2021-02-23 14:18:26dtom9424setnosy: + dtom9424
type: crash -> security
messages: + msg387571
2021-02-22 04:37:36vkisforeversetmessages: + msg387498
2021-02-22 04:32:11anthonypjshawsetnosy: + anthonypjshaw
messages: + msg387497
2021-02-22 04:22:11vkisforeversetnosy: - anthonypjshaw
messages: + msg387496
2021-02-22 04:21:13anthonypjshawsetnosy: + anthonypjshaw
messages: + msg387495
2021-02-22 04:11:29vkisforevercreate