classification
Title: xml.etree.ElementTree: Element.append doesn't type-check its argument
Type: behavior Stage: resolved
Components: Library (Lib), XML Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: effbot, eli.bendersky, flox, python-dev, santa4nt, sjmachin, terry.reedy, tshepang
Priority: normal Keywords: patch

Created on 2012-01-14 00:54 by sjmachin, last changed 2012-03-23 12:27 by eli.bendersky. This issue is now closed.

Files
File name Uploaded Description Edit
issue13782.1.patch eli.bendersky, 2012-03-17 06:13 review
Messages (11)
msg151210 - (view) Author: John Machin (sjmachin) Date: 2012-01-14 00:54
import xml.etree.ElementTree as et
node = et.Element('x')
node.append(not_an_Element_instance)

2.7 and 3.2 produce no complaint at all.
2.6 and 3.1 produce an AssertionError.

However cElementTree in all 4 versions produces a TypeError.

Please fix 2.7 and 3.2 ElementTree to produce a TypeError.
msg151233 - (view) Author: Santoso Wijaya (santa4nt) * Date: 2012-01-14 06:06
This does not only affect `append`, but also, `insert`, `extend`, etc.

In fact, the list-like operations on Element are merely forwarded to its `_children` (a list) field. Should we really type check these methods' arguments each? Doesn't strike as Pythonic to me...

OTOH, in cElementTree, by virtue of C-API's tuple unpacking convention, the argument's type is always strictly checked prior to accessing the children list.
msg151708 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-01-20 23:18
Confirmed with on 3.2.2 Win7 for all three methods.
Docs say .append should raise AssertionError (see below).
However, that is unusual and TypeError is normal practice.
I am not sure what to do.

Our docs say: '''
append(subelement) 
Adds the element subelement to the end of this elements internal list of subelements.

extend(subelements) 
Appends subelements from a sequence object with zero or more elements. Raises AssertionError if a subelement is not a valid object.

insert(index, element) 
Inserts a subelement at the given position in this element.
'''

This implies to me that append and insert should do the same.
In fact, the current code has the assertion checks but they are commented out in all methods.

 From
http://effbot.org/zone/pythondoc-elementtree-ElementTree.htm
"append ... Raises AssertionError:
    If a sequence member is not a valid object."
Ditto for .insert

Florent: why are all the assertion checks commented out in
Changeset: 59511 (831e30c0fd73) Issue #6472:
The xml.etree package is updated to ElementTree 1.3.
(March 2010)

Is the change in Fredrik's code, in spite of his docs?
msg152989 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-09 21:36
Actuallly, the assertions were commented in the upstream repository.

https://bitbucket.org/effbot/et-2009-provolone/src/9e9e7f2710aa/elementtree/CHANGES#cl-46

> (1.3a6 released)
> - Removed most assert statements from the Element and ElementTree code.


It looks more useful to me to have a consistent behavior between C and Python version.
msg153008 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 03:38
>> It looks more useful to me to have a consistent behavior between C and Python version.

I agree, especially is we now consider to expose the C API by default, leaving the Python API as a fallback.
msg153024 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-02-10 05:13
Option 1 is to fix the regression in the 2.7/3.2 Python versions and change the C version to conform to the doc and older Python versions., even though it is the wrong error, and even though Python asserts disappear with -O (which is why it is the wrong error).

Option 2 is to leave cET alone with the correct error and fix 2.7/3.2 and the docs to conform.

Option 3 is to leave cET alone and leave 2.7/3.2 alone as done deals, even if bad. Change 3.3 Python and docs to TypeError. We would need a version-changed note, but perhaps should mention the 3.2 behavior so as to not give a mis-impression.

I think I favor 3, but after presenting the trilemma on pydev.
msg153025 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 05:18
Terry, I agree with 3. The "regression" here is insignificant enough in my view to warrant mucking with already released versions.
msg156136 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-03-17 04:42
Link to related python-dev discussion: http://mail.python.org/pipermail/python-dev/2012-March/117715.html
msg156141 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-03-17 06:12
Attaching a patch that sets the record straight in 3.3 - the 3 methods will raise TypeError, in both C and Python implementations. Also adds a test that verifies this, and updating the documentation.

Note that in the C implementation extend wasn't actually type-checking its argument as well, so I added it for consistency. I don't believe this will be a performance problem. On the contrary, there are some type checking calls inside the implementation that can probably be removed now that we make sure only subclasses of Element get in as children. I'll look into this later.
msg156650 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-23 12:24
New changeset 75268a773455 by Eli Bendersky in branch 'default':
Issue #13782: streamline argument type-checking in ET.Element
http://hg.python.org/cpython/rev/75268a773455
msg156651 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-03-23 12:26
Closing the issue.

3.3 was fixed, and there was no concensus regarding 2.7/3.2 in the list, so status-quo as usual.
History
Date User Action Args
2012-03-23 12:27:11eli.benderskysetstage: test needed -> resolved
2012-03-23 12:26:57eli.benderskysetstatus: open -> closed
resolution: fixed
messages: + msg156651
2012-03-23 12:24:53python-devsetnosy: + python-dev
messages: + msg156650
2012-03-17 06:13:52eli.benderskysetfiles: + issue13782.1.patch
2012-03-17 06:13:40eli.benderskysetfiles: - issue13782.1.patch
2012-03-17 06:12:30eli.benderskysetfiles: + issue13782.1.patch
keywords: + patch
messages: + msg156141
2012-03-17 04:42:46eli.benderskysetmessages: + msg156136
2012-02-10 05:18:33eli.benderskysetmessages: + msg153025
2012-02-10 05:13:50terry.reedysetmessages: + msg153024
2012-02-10 03:38:22eli.benderskysetmessages: + msg153008
2012-02-09 21:36:23floxsetmessages: + msg152989
2012-02-08 09:13:25tshepangsetnosy: + tshepang
2012-02-08 03:44:33eli.benderskysetnosy: + eli.bendersky
2012-01-20 23:18:28terry.reedysetnosy: + terry.reedy, effbot, flox

messages: + msg151708
stage: test needed
2012-01-14 06:06:40santa4ntsetmessages: + msg151233
2012-01-14 03:33:16santa4ntsetnosy: + santa4nt

components: + Library (Lib), XML
versions: + Python 3.3, - Python 2.6, Python 3.1
2012-01-14 00:54:12sjmachincreate