Title: xml.ElementTree iterparse filehandle left open
Type: resource usage Stage:
Components: XML Versions: Python 3.9
Status: open Resolution:
Dependencies: Superseder: Add the close method for ElementTree.iterparse() object
View: 25707
Assigned To: Nosy List: anthonypjshaw, dtom9424, martin.panter, vkisforever
Priority: normal Keywords:

Created on 2021-02-22 04:11 by vkisforever, last changed 2021-03-10 20:15 by martin.panter.

File name Uploaded Description Edit vkisforever, 2021-02-22 04:11 iterparse issue recreator
Messages (7)
msg387494 - (view) Author: Viktor Kis (vkisforever) * Date: 2021-02-22 04:11
Standard library xml.ElementTree - iterparse does not close filehandle properly when breaking out of a for loop iterating with iterparse. The issue only occurs when the user passes a "string" as a source object instead of a filehandle (that is because a user's filehandle can safely be closed in a context manager as shown in the attachment).
msg387495 - (view) Author: anthony shaw (anthonypjshaw) * (Python triager) Date: 2021-02-22 04:21
which version of Python were you doing this on?

The function in question is
msg387496 - (view) Author: Viktor Kis (vkisforever) * Date: 2021-02-22 04:22
ran python 3.9.2 on windows 10

1) What did you do?
Tried to remove folder that contains a xml file that was worked on by ElementTree iterparse with argument source='filename' (string type input)
2. What happened?
python was not able to remove the folder due to file lock still held by iterparse.
3. What did you expect to happen
iterparse was supposed to close the filehandle such that the user could then write/move/delete the xml file worked on, however python gave a PermissionError:

Traceback (most recent call last):
  File "C:\Users\vkisf\Desktop\00_MYSTUFF\01_SOFTWARE\01_PYTHON\01_GITHUB\pycascades2021\", line 43, in <module>
  File "C:\Users\vkisf\Desktop\00_MYSTUFF\01_SOFTWARE\01_PYTHON\01_GITHUB\pycascades2021\cpython\lib\", line 740, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\vkisf\Desktop\00_MYSTUFF\01_SOFTWARE\01_PYTHON\01_GITHUB\pycascades2021\cpython\lib\", line 618, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\vkisf\Desktop\00_MYSTUFF\01_SOFTWARE\01_PYTHON\01_GITHUB\pycascades2021\cpython\lib\", line 616, in _rmtree_unsafe
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'subdir2\\app.xml'
msg387497 - (view) Author: anthony shaw (anthonypjshaw) * (Python triager) Date: 2021-02-22 04:32
Example script attached works perfectly on macOS 

> python3.9
[('default', ''), ('vt', '')]
[('default', ''), ('vt', '')]
msg387498 - (view) Author: Viktor Kis (vkisforever) * Date: 2021-02-22 04:37
Confirmed that no apps were using the created xml file or folder. Windows Explorer was closed for all folders only the terminal shell was open. Issue still persists on windows 10 OS.
msg387571 - (view) Author: Tom Dougherty (dtom9424) Date: 2021-02-23 14:18
"erase all files"
msg388457 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2021-03-10 20:14
Perhaps this can be handled with Issue 25707, which is open for adding an API to close the file, similar to how "os.scandir" iterator implements a context manager and "close" method.
Date User Action Args
2021-03-10 20:15:23martin.pantersettype: security -> resource usage
2021-03-10 20:14:54martin.pantersetsuperseder: Add the close method for ElementTree.iterparse() object

messages: + msg388457
nosy: + martin.panter
2021-02-23 14:18:26dtom9424setnosy: + dtom9424
type: crash -> security
messages: + msg387571
2021-02-22 04:37:36vkisforeversetmessages: + msg387498
2021-02-22 04:32:11anthonypjshawsetnosy: + anthonypjshaw
messages: + msg387497
2021-02-22 04:22:11vkisforeversetnosy: - anthonypjshaw
messages: + msg387496
2021-02-22 04:21:13anthonypjshawsetnosy: + anthonypjshaw
messages: + msg387495
2021-02-22 04:11:29vkisforevercreate