classification
Title: xml.etree.ElementTree.extend does not work with iterators when using the Python implementation
Type: behavior Stage: patch review
Components: XML Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: alexprengere, eli.bendersky, lukasz.langa, miss-islington, rhettinger, scoder, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2021-03-04 15:59 by alexprengere, last changed 2021-03-30 21:36 by miss-islington.

Pull Requests
URL Status Linked Edit
PR 24751 merged alexprengere, 2021-03-04 16:36
PR 25101 merged miss-islington, 2021-03-30 21:12
PR 25102 merged miss-islington, 2021-03-30 21:12
Messages (9)
msg388099 - (view) Author: Alex (alexprengere) * Date: 2021-03-04 15:59
This issue is only visible when the C accelerator of ElementTree is *not* used. It is the counterpart of the following issue on PyPy3: https://foss.heptapod.net/pypy/pypy/-/issues/3181

 >>> from xml.etree.ElementTree import Element
 >>> r = Element("root")
 >>> r.extend((Element(str(i)) for i in range(3)))
 >>> print(list(r))
 []

When using the C accelerator, the list is not empty, as expected. In the Python code, a check on the input empties the input iterator.

The fix is trivial (one-line change), so if you are interested I could open a PR, which would be my first, so a good occasion to go through the devguide ;)
I understand that since Python3.3 the C accelerator is used by default, so I would agree that this is not really a bug, and I can just fix it on PyPy side.
msg388118 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-03-04 18:32
Your proposed fix LGTM. Please open a PR. Don't forget about test and NEWS entry.
msg388119 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2021-03-04 18:39
Thanks for the report and the PR. I would argue that this can still be fixed in Py3.8, given the low impact (but not earlier). The behaviour is clearly wrong (in a subtle way) and most users won't notice it because they won't switch off the C accelerator module.
msg388137 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-03-04 23:29
+1 for back porting, but check with the RM first.
msg389830 - (view) Author: Alex (alexprengere) * Date: 2021-03-30 15:20
Is there anything else I should be doing, in order to get this merged in time for 3.10? (the PR is "awaiting merge")
For the record, the equivalent PR for PyPy is now merged in the py3.7 branch: https://foss.heptapod.net/pypy/pypy/-/merge_requests/807.

Regarding the backporting, I understand from the devguide this is done automatically from GitHub labels (nice!), but it does not look like I have the rights to add those to my PR.
msg389853 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-03-30 21:11
New changeset 51a85ddce8b336addcb61b96f04c9c5edef07296 by Alex Prengère in branch 'master':
bpo-43399: Fix ElementTree.extend not working on iterators (GH-24751)
https://github.com/python/cpython/commit/51a85ddce8b336addcb61b96f04c9c5edef07296
msg389856 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-03-30 21:23
Pinging us was the only action needed. Thank you for your contribution Alex.
msg389857 - (view) Author: miss-islington (miss-islington) Date: 2021-03-30 21:33
New changeset c1079cde2a7676892a9b98703903206b7d26ed1f by Miss Islington (bot) in branch '3.8':
bpo-43399: Fix ElementTree.extend not working on iterators (GH-24751)
https://github.com/python/cpython/commit/c1079cde2a7676892a9b98703903206b7d26ed1f
msg389858 - (view) Author: miss-islington (miss-islington) Date: 2021-03-30 21:36
New changeset b500bd8e672d15c6dfa24568a3264fdc0f3e0c01 by Miss Islington (bot) in branch '3.9':
bpo-43399: Fix ElementTree.extend not working on iterators (GH-24751)
https://github.com/python/cpython/commit/b500bd8e672d15c6dfa24568a3264fdc0f3e0c01
History
Date User Action Args
2021-03-30 21:36:33miss-islingtonsetmessages: + msg389858
2021-03-30 21:33:03miss-islingtonsetmessages: + msg389857
2021-03-30 21:23:22serhiy.storchakasetmessages: + msg389856
2021-03-30 21:12:46miss-islingtonsetpull_requests: + pull_request23847
2021-03-30 21:12:35miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request23846
2021-03-30 21:11:49serhiy.storchakasetmessages: + msg389853
2021-03-30 15:20:15alexprengeresetmessages: + msg389830
2021-03-04 23:41:57rhettingersetnosy: + lukasz.langa
2021-03-04 23:29:56rhettingersetnosy: + rhettinger
messages: + msg388137
2021-03-04 18:39:15scodersetmessages: + msg388119
versions: - Python 3.6, Python 3.7
2021-03-04 18:32:45serhiy.storchakasetmessages: + msg388118
2021-03-04 16:43:32xtreaksetnosy: + scoder, eli.bendersky, serhiy.storchaka
2021-03-04 16:36:48alexprengeresetkeywords: + patch
stage: patch review
pull_requests: + pull_request23521
2021-03-04 15:59:21alexprengerecreate