classification
Title: Expose the C implementation of ElementTree by default when importing ElementTree
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, cmn, effbot, eli.bendersky, eric.araujo, ezio.melotti, flox, pjenvey, python-dev, scoder, tshepang
Priority: normal Keywords: patch

Created on 2012-02-10 14:52 by eli.bendersky, last changed 2012-06-15 04:49 by eli.bendersky. This issue is now closed.

Files
File name Uploaded Description Edit
issue13988_prepare_pep399.diff flox, 2012-02-10 21:39 review
issue13988_prepare_pep399_v2.diff flox, 2012-02-11 09:37 review
issue13988_fold_cET_behind_ET.diff flox, 2012-02-11 10:39 review
issue13988_doc_news.1.patch eli.bendersky, 2012-02-12 04:21 review
issue13988_fold_cET_behind_ET_v2.diff flox, 2012-02-12 12:52 review
findall_takes_no_keywords_anymore.py cmn, 2012-05-15 18:14
Messages (59)
msg153052 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 14:52
Following the discussion on python-dev [1], this issue will track the re-organization of Lib/xml/etree to expose the C implementation (_elementtree) by default when importing ElementTree. The test suite will also have to be updated - it's required that it exercises both the C and the Python implementations.

I would like to make the import "magic" simple. Thus, the idea I currently plan to pursue is:

* xml/etree/ElementTree.py will be a simple facade that attempts to 'import *' from _elementtree, and on failure does 'import *' from pyElementTree
* The current contents of xml/etree/ElementTree.py will move to xml/etree/pyElementTree.py
* xml/etree/cElementTree.py disappears.

The test suite will be modified accordingly.


I'll be working on creating a patch for this. Any help, ideas, comments and discussions are more than welcome.


[1] http://mail.python.org/pipermail/python-dev/2012-February/116278.html
msg153053 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 14:54
Oh, and not to forget: the documentation has to be updated to just not mention cElementTree any longer. For the user, the fact that a fast C library is invoked underneath should be invisible.
msg153056 - (view) Author: Stefan Behnel (scoder) * Date: 2012-02-10 15:37
Eli Bendersky, 10.02.2012 15:52:
> * The current contents of xml/etree/ElementTree.py will move to xml/etree/pyElementTree.py

IIRC, there is a well specified way how accelerator modules should be used
by Python modules. I recall a lengthy discussion on python-dev (or the py3k
list?) back in the old pre-3.0 days, maybe there's even a PEP?

> * xml/etree/cElementTree.py disappears.

Careful with backwards compatibility here. It's the accelerator module
(_elementtree.so, IIRC) which is to be moved behind ElementTree.py.

I don't see a compelling enough reason to break imports in existing code by
removing the cElementTree module, so we should not do that.

Stefan
msg153057 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 15:43
>> IIRC, there is a well specified way how accelerator modules should be used
by Python modules. I recall a lengthy discussion on python-dev (or the py3k
list?) back in the old pre-3.0 days, maybe there's even a PEP?

If there's a convention, I'll happily follow it. It would be great if someone could dig up the relevant details (I'll try to look for them myself).

>> I don't see a compelling enough reason to break imports in existing code by
removing the cElementTree module, so we should not do that.

Agreed. Perhaps it should just be deprecated?
msg153058 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 15:46
Hmm, that may be PEP 399:

  If an acceleration module is provided it is to be named the same as the module it is accelerating with an underscore attached as a prefix, e.g., _warnings for warnings. The common pattern to access the accelerated code from the pure Python implementation is to import it with an import *, e.g., from _warnings import *. This is typically done at the end of the module to allow it to overwrite specific Python objects with their accelerated equivalents. 

However, it's hardly a rule, just describing a "common pattern". I wonder why this approach is preferable to the one I proposed (explicit facade module)?
msg153059 - (view) Author: Stefan Behnel (scoder) * Date: 2012-02-10 15:55
Eli Bendersky, 10.02.2012 16:43:
>>> I don't see a compelling enough reason to break imports in existing code by
>>> removing the cElementTree module, so we should not do that.
> 
> Agreed. Perhaps it should just be deprecated?

Given that its mere existence is currently almost undocumented (except for
one tiny sentence in the docs), I vote for clearly documenting it as
deprecated, yes, with a mention to the fact that it's automatically used by
xml.etree.ElementTree starting with 3.3.

I wouldn't want to see more than that done, though, specifically not a
visible warning when it's being imported. There's far too much code out
there that uses it in previous Python versions. Such a warning would strike
even if it's only being imported conditionally with a try-except, which is
the most common way of doing it. So it would hit most users and require an
awful lot of code to be touched to fix it, for basically zero benefit.

Stefan
msg153061 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-02-10 16:07
A note in the doc is easy to miss IMHO, and since DeprecationWarnings are silenced by default, I don't think they will affect the final users.

A different "problem" is that developers will have to check for the Python version if they want to use ElementTree on Python >=3.3 and keep using cElementTree on <3.3 (unless another way is provided).

If possible I would avoid pyElementTree, and keep ElementTree that imports from _elementtree and the deprecated cElementTree (until it can be removed).
msg153062 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 16:12
> If possible I would avoid pyElementTree,

I suppose it's possible, but I'm genuinely interested in a technical reason for doing so. The approach suggested in PEP 399 is useful for module in which part of the functionality is implemented in C, and then augmented in Python. ElementTree is different - it's pretty much two separate implementations of the same API.

So, I think there's little question in terms of simplicity and clarity. Having pyElementTree and cElementTree (keeping it for backwards compat), and a facade named ElementTree that chooses between them is simple, clean and intuitive.

From a performance point of view, consider the (by far) common case - where _elementtree *is* successfully imported. 

Option 1: from _elementtree import *, at the end of the Python implementation in ElementTree.py - so for each invocation, the whole import of the Python code has to be done, just to reach the overriding import * at the end.

Option 2: ElementTree is a facade that attempts to import _elementtree first. So the Python implementation in pyElementTree doesn't even have to be parsed and imported
msg153063 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-10 16:30
> If possible I would avoid pyElementTree,

Me too:
 - __name__ and __qualname__ would be less confusing
 - the cElementTree accelerator uses large parts of Python implementation

> ElementTree is different - it's pretty much two separate implementations of the same API.

Not fully separated... there's some python code hidden in the C module.

> From a performance point of view, consider the (by far) common case
> - where _elementtree *is* successfully imported.
> ... for each invocation, the whole import of the Python code has
> to be done, just to reach the overriding import * at the end.

This point is wrong... the _elementtree.c accelerator imports Python ElementTree already.

As you can see on lines 2938 to 2945, the change could lead to an import cycle:
http://hg.python.org/cpython/file/705b56512287/Modules/_elementtree.c#l2938

Trying to sort this out, it already gives me a headache.
I would like to remove the Python bootstrap code from the C module and try to do it differently, in a more standard way.
msg153064 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 16:37
>> From a performance point of view, consider the (by far) common case
>> - where _elementtree *is* successfully imported.
>> ... for each invocation, the whole import of the Python code has
>> to be done, just to reach the overriding import * at the end.
>
> This point is wrong... the _elementtree.c accelerator imports Python ElementTree already.
>
> As you can see on lines 2938 to 2945, the change could lead to an import cycle:
> http://hg.python.org/cpython/file/705b56512287/Modules/_elementtree.c#l2938
>
> Trying to sort this out, it already gives me a headache.
> I would like to remove the Python bootstrap code from the C module and try to do it differently, in a more standard way.
>

The Python code inside _elementtree could be moved to Python code,
which would then import the Python stuff it needs from pyElementTree.
Since pyElementTree doesn't import _elementtree, there will be
circular dependencies.

So this is a point *in favor* of pyElementTree being pure-Python :-)

In other words:

In xml/etree there is:

- ElementTree: tries to import cElementTree. On success, done. On
ImportError, imports pyElementTree
- pyElementTree: the pure Python implementation
- cElementTree: sets up the bootstrap Python code and tries to import
_elementtree. In case of an error, propagates an ImportError up.

Would that work?
msg153065 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 16:38
Oops, in last message:

s/there will be circular dependencies/there will not be circular dependencies/
msg153066 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-02-10 16:46
> In xml/etree there is:
> - ElementTree: tries to import cElementTree. On success, done. On
>   ImportError, imports pyElementTree
> - pyElementTree: the pure Python implementation
> - cElementTree: sets up the bootstrap Python code and tries to import
>   _elementtree. In case of an error, propagates an ImportError up.

What I had in mind is more like:
 - ElementTree: defines the python code and if _elementtree is available overrides part of it with the functions imported from it;
 - cElementTree: at this point it could just be a deprecated alias for ElementTree
msg153068 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-10 17:26
> What I had in mind is more like:
>  - ElementTree: defines the python code and if _elementtree is available overrides part of it with the functions imported from it;

The problem with this is the bootstrap Python code executed by
_elementtree. That should not be executed when _elementtree (the C
parts) can't be imported. Keeping this code in ElementTree will
probably complicate matters since it will add import conditions.
msg153075 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-02-10 20:43
>>  - ElementTree: defines the python code and if _elementtree is 
>>  available overrides part of it with the functions imported from it;

> The problem with this is the bootstrap Python code executed by
_elementtree.

This might become unnecessary if ElementTree becomes the main module and _elementtree only contains a few faster functions/classes that are supposed to replace the ones written in Python.
So basically you only have a single fully functional Python module (ElementTree) plus an optional C module (_elementtree) that only provides faster replacements for ElementTree.

> That should not be executed when _elementtree (the C parts) can't be
> imported.

We are assuming that _elementtree might be missing, but what are the cases where this might actually happen? Other implementations like PyPy? Exotic platforms that can't compile _elementtree?

> Keeping this code in ElementTree will probably complicate 
> matters since it will add import conditions.

Wouldn't that as simple as having in ElementTree.py:
...
full python code here...
...
try:
    # override a few functions/classes with the faster versions
    from _elementtree import *
except ImportError:
    # _elementtree is missing, so we just keep the "slow" versions
    pass
else:
    # do the rest here if at all needed (e.g. plug the faster
    # versions in the right places)

I'm not familiar with ElementTree (I just looked at the bootstrap bit quickly), so what I'm saying might not be applicable here, but I've seen other modules doing something similar to what I'm proposing (json, heapq, maybe even warning and others).
msg153078 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-10 21:39
The first step is to strip out the cElementTree bootstrap code from the C module.
I did it in the attached patch (plus removal of obsolete code for copy() in Python 2.4).
This passes the unmodified tests "test_xml_etree" and "test_xml_etree_c".

Then I think the right approach is to fold completely cElementTree behind ElementTree.
The cElementTree alias can be simply declared in Lib/xml/etree/__init__.py.
msg153086 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-11 03:34
Ezio,

>> We are assuming that _elementtree might be missing, but what are the cases where this might actually happen? Other implementations like PyPy? Exotic platforms that can't compile _elementtree?

I guess both. To make the stdlib work on PyPy without changes, it has to be able to load the pure Python modules in a fallback.

As for platforms that can't compile _elementtree, keep in mind that there's also expat which _elementtree uses, so it's a lot of code to compile. Python works on some embedded systems, I'm not sure all of them can compile this code.
msg153089 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-11 04:05
Florent, thanks for the patch - at this point code is more useful than talk :-)

Anyhow, I tried to apply it and a few tests in test_xml_etree_c fail, because it can't find fromstring and fromstringlist. This gets fixed when I import fromstringlist in cElementTree.py from ElementTree, and in the same file assign:

  fromstring = XML

Which is similar to what ElementTree itself does.

In general, I agree that a good first step would be to refactor the code to extract the boostrapping from _elementtree.c to cElementTree.py. As long as the tests pass, this can be committed regardless of this issue's original intent.

However, why did you leave some bootstrapping code inside? Can't all of it go away?
msg153101 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-11 05:32
I strongly feel that existing code importing ElementTree or cElementTree should not be broken.  Let’s add transparent import from _elementtree to ElementTree without breaking existing uses of cET.

I think that 3.2 and 2.7 should get a doc note about cET, do we have a bug for this?
msg153109 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-11 07:41
> I strongly feel that existing code importing ElementTree or cElementTree should not be broken.  Let’s add transparent import from _elementtree to ElementTree without breaking existing uses of cET.
>

AFAICS there's currently no disagreement on this point. The import
from cElementTree will keep working in 3.3 as it always had. However,
the explicit mention of cElementTree should be removed from the
documentation of ElementTree. The only remaining question is whether a
silent deprecation warning should be added in cElementTree.

> I think that 3.2 and 2.7 should get a doc note about cET, do we have a bug for this?
>

What doc note? Something in the spirit of: "Note that in 3.3, the
accelerated C implementation will be provided by default when
importing ElementTree" - or do you mean something else?

I don't think there's an open bug for this.
msg153110 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-11 08:08
The more I think about it, the more the bootstrap code in _elementtree.c annoys me. It's the only instance of calling PyRun_String in Modules/ !

It's hackish and causes ugly import problems. If the C code needs stdlib functionality like copy.deepcopy, it should use PyImport_ImportModule like everyone else and not through a PyRun_String hack.

Since we've already decided to do some refactoring, I suggest all trace of the bootstrap is removed from _elementtree.c
msg153111 - (view) Author: Stefan Behnel (scoder) * Date: 2012-02-11 08:39
Eli Bendersky, 11.02.2012 09:08:
> The more I think about it, the more the bootstrap code in _elementtree.c
> annoys me. It's the only instance of calling PyRun_String in Modules/ !
> 
> It's hackish and causes ugly import problems. If the C code needs stdlib
> functionality like copy.deepcopy, it should use PyImport_ImportModule
> like everyone else and not through a PyRun_String hack.

I find it perfectly legitimate to run Python code from a C module.
Certainly not a hack. We all know that most non-trivial functionality can
be expressed much easier in Python than in C, that's why we use Python
after all. In particular, defining a class with attributes and methods is a
couple of lines of code in Python, but a huge amount of code in C. Avoiding
the complexity of writing everything in C, or even of splitting the code in
a harder to understand way, is worth it.

That being said, I think it's worth removing any clear *redundancy* with
ET.py, as Florent's patch did. The goal is to keep _elementtree.c a pure
accelerator module that improves plain ElementTree, and redundancy is
counterproductive in this context. But if the implementation differs for
some reason, I would tend towards leaving it as is.

Stefan
msg153112 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-11 08:58
>> I find it perfectly legitimate to run Python code from a C module.
Certainly not a hack. We all know that most non-trivial functionality can
be expressed much easier in Python than in C, that's why we use Python
after all. In particular, defining a class with attributes and methods is a couple of lines of code in Python, but a huge amount of code in C. Avoiding the complexity of writing everything in C, or even of splitting the code in a harder to understand way, is worth it.
<<

There can be arguments both way, but if we follow the lead of existing standard extension modules, the tendency is clearly not to use PyRun_String. Many C extensions use functionality from Python, but none does it the "bootstrap way". Why is that? Is there a good reason, or is it just convention?
msg153114 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-11 09:23
> Anyhow, I tried to apply it and a few tests in test_xml_etree_c fail,
> because it can't find fromstring and fromstringlist.

Ooops, I cut some redundancy after running the tests, and I forgot to re-add the import. You're right.

> However, why did you leave some bootstrapping code inside?
> It's the only instance of calling PyRun_String in Modules/

I just tried to cut the import cycle and import it the other way.
I think it was done like that historically, for some reason, when
the module was first developped (for Python 1.5 maybe ...)
It is not necessary to remove all the Python code at once, and I am better at Python than at C.
We can delay this additional clean-up at a later time, it does not
block the PEP399 implementation.
msg153115 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-11 09:35
Updated patch:
 - fixed missing import and missing alias
 - moved the XMLTreeBuilder alias to the Python module
msg153119 - (view) Author: Roundup Robot (python-dev) Date: 2012-02-11 10:29
New changeset 31dfb4be934d by Florent Xicluna in branch 'default':
Issue #13988: move the python bootstrap code to cElementTree.py, and remove obsolete code for Python 2.4 and 2.5.
http://hg.python.org/cpython/rev/31dfb4be934d
msg153120 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-11 10:39
I've pushed this first part, which is just a code refactoring.

I tried to work out a patch for the second part.
The tricky thing is because of xml.etree still using doctests.
The patch for the tests seems to be enough small and readable.

We have small differences between C and Python, about the warnings beeing raised. In general the C implementation do not raise the deprecation warnings. IMHO, this could be fixed later.

Still missing is the patch for the documentation.
msg153122 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-11 11:59
Another random cleanup idea:

ElementTree.py has this code:

try:
    from . import ElementPath
except ImportError:
    ElementPath = _SimpleElementPath()

Since in the stdlib ElementPath.py is always there, this is meaningless, so I'd say this try... except ImportError contraption can be removed, as well as _SimpleElementPath, and just replaced by:

from . import ElementPath
msg153151 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-12 03:44
>> I think that 3.2 and 2.7 should get a doc note about cET
> What doc note?

I was referring to one of the points raised in the email thread: the docs don’t tell people that they can import a faster ET version, cET.
msg153155 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-12 03:49
>> I was referring to one of the points raised in the email thread: the docs don’t tell people that they can import a faster ET version, cET.

Well, they *do*, but very modestly :-) I agree that should be improved and emphasized a bit, perhaps even mentioning (at least in the 3.2 doc) that in 3.3 it's going to be done by default. Is that accepted practice? I mean to mention in docs of 3.x some upcoming change in version 3.x+1 ?
msg153158 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-12 04:12
Florent,

The issue13988_fold_cET_behind_ET.diff patch looks good to me. Unless there are objections from others, you can commit! Even the tests became simpler now, that's awesome ;-)

Now, what's left for this issue:

1. Update the documentation
2. Update "what's new"
3. Discuss and possibly implement the deprecation of cElementTree

I'll send a patch for 1 & 2 soon. What about 3?
msg153160 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-12 04:21
Attaching a patch for Doc/library/xml.etree.elementtree.rst and Misc/NEWS.

The doc notice is modeled after a similar notice in the doc of 'pickle'. Note that I've also removed the mention that effbot's site is the home of the development version of the library, since it's no longer formally true.
msg153165 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-12 04:35
Florent,

Just something minor I noticed in the new cElementTree.py:

  # Wrapper module for _elementtree

  from xml.etree.ElementTree import *
  # Not in __all__
  from xml.etree.ElementTree import ElementPath, XMLID, register_namespace

The "wrapper" comment should be removed, and possibly replaced with "This module exists for backwards compatibility with releases earlier than 3.3; Please import xml.etree.ElementTree directly"

Also, regarding the names not in __all__. Any good reason for them not to be? Seems like an omission to me.
msg153175 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-12 05:11
> perhaps even mentioning (at least in the 3.2 doc) that in 3.3 it's going
> to be done by default. Is that accepted practice?
I don’t think we ever do that, and it would be confusing.

-1 to mentioning _elementtree, an implementation detail.  What I was talking about is cElementTree.
msg153180 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-12 05:46
Éric,

Could you please open a new issue (with a dependency on this one) and explain there clearly what you want/mean?
msg153203 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-12 12:28
There was a discussion in December which ruled out some CPython too specific implementation details from the documentation.
http://mail.python.org/pipermail/python-dev/2011-December/115063.html

Maybe it's better to remove these 2 lines about the "transparent optimizer". Then the "versionchanged" tag can be changed a little:
.. versionchanged:: 3.3
   This module will use a fast implementation whenever available.
   The module :mod:`xml.etree.cElementTree` is deprecated.

Probably we'll add few words in the Doc/whatsnew/3.3.rst too.
msg153204 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-12 12:52
Updated patch:
 - add 'XMLID' and 'register_namespace' to the ElementTree.__all__
 - the comment says explicitly that cElementTree is deprecated
 - exercise the deprecated module with the tests
msg153221 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-12 19:06
Florent,

Your updated patch looks good. I think that the explicit import of _namespace_map into cElementTree is just to satisfy some weird magic in the tests and can probably be removed as well (along with the weird magic :-), but that's not really important and can be left for later cleanups.

Regarding the documentation, alright let's not mention the implementation detail, and your "versionchanged" addition makes sense. I don't think adding directly to whatsnew/3.3.rst is necessary, updating Misc/NEWS is enough.

I'll apply the documentation patch after you apply the code patch. Or if you want, you can apply it yourself, I don't mind.

Thanks for the cooperation!
msg153223 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-12 19:11
By the way, I see that if the explicit import of _namespace_map is commented out, the test_xml_etree_c test fails because it's not in the __all__ list. So the test can just import it directly with:

from xml.etree.ElementTree import _namespace_map

And the import in cElementTree won't be necessary. After all, _namespace_map is definitely not a public API!

This will keep cElementTree an nice-and-clean:

from xml.etree.ElementTree import *
msg153230 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-12 21:07
> from xml.etree.ElementTree import _namespace_map
>
> And the import in cElementTree won't be necessary.
> After all, _namespace_map is definitely not a public API!

Because of the interaction of the support.import_fresh_module with the CleanContext context manager, it's not so easy to remove black magic.
I don't find better than:

        if hasattr(ET, '_namespace_map'):
            _namespace_map = ET._namespace_map
        else:
            from xml.etree.ElementTree import _namespace_map


This is why I kept the import in the deprecated "cElementTree" at first.
It does not hurt (it's private API), and it makes the test easier.

( If you have doubts, try ./python -m test test_xml_etree{,_c} or variants. )


I will probably commit code and documentation at once. It makes things easier regarding traceability.
msg153249 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-13 04:25
Alright, it's not really important at this point and can be cleaned up
later.

>
> I will probably commit code and documentation at once. It makes things
> easier regarding traceability.
>

Sounds good
msg153255 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-02-13 09:18
FWIW the JSON doc doesn't even mention the acceleration module _json, but since people here are used to import cElementTree I think it should be mentioned that it's now deprecated and accelerations are used automatically, so something like this would work:

.. versionchanged:: 3.3
   The :mod:`xml.etree.cElementTree` module is now deprecated.
   A fast implementation will be used automatically whenever available.

I also agree with Éric that there's no need to mention _elementtree (people might try to import that instead, and other implementations might use a different name).

Lib/test/test_xml_etree_c.py could also be removed, and the other tests could import cElementTree too (even though I'm not sure that works too well with doctests).

Shouldn't cElementTree raise an error when _elementtree is missing?
A DeprecationWarning should be added too.
msg153258 - (view) Author: Roundup Robot (python-dev) Date: 2012-02-13 10:04
New changeset 65fc79fb4eb2 by Florent Xicluna in branch 'default':
Issue #13988: cElementTree is deprecated and the _elementtree accelerator is automatically used whenever available.
http://hg.python.org/cpython/rev/65fc79fb4eb2
msg153259 - (view) Author: Roundup Robot (python-dev) Date: 2012-02-13 11:15
New changeset e9cf34d56ff1 by Florent Xicluna in branch 'default':
Fix xml_etree_c test error (follow up of issue #13988).
http://hg.python.org/cpython/rev/e9cf34d56ff1
msg153265 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-02-13 12:34
Now the merge is done. Thank you Eli for the effort, and to the other contributors for the review.

Following topics may need further work:

 - add a Deprecation warning for cElementTree? it will annoy the package maintainers which support both 3.2 and >= 3.3, because either they'll use the non-accelerated version in 3.2, or they will have the Deprecation warning in 3.3 ... IMHO, it's better to do nothing, and just keep the mention in the documentation that it is deprecated.

 - raise the Deprecation warnings for the functions and methods which are marked as deprecated in the documentation (the Python code does it, but not the C accelerator)

 - convert _elementtree.c Python bootstrap code to C

 - refactor the test suite


These topics are not high priority. A specific issue should be opened if any of them require some attention.
msg153319 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-14 03:31
I would add to the TODO - improve the documentation of the module. Opened issue 14006 for this.
msg153321 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-14 03:56
I started going over the deprecated methods in ElementTree and ran into a more serious problem. XmlParser.doctype() is listed as deprecated, and indeed ElementTree (the Python version) issues a deprecation warning. However, the C implementation doesn't have doctype() at all so I get AttributeError.
msg153323 - (view) Author: Philip Jenvey (pjenvey) * (Python committer) Date: 2012-02-14 04:08
DeprecationWarnings aren't that annoying anymore now that they're silent by default. It should at least have a PendingDeprecationWarning
msg153326 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-14 04:19
Opened issue 14007 to track the doctype() problem
msg153453 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-02-16 03:29
Emitting a deprecation warning on importing cElementTree has been rejected in the pydev list. The other remaining tasks have new issues on them, so this issue is done now.
msg153480 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-02-16 13:04
I'm still not sure that's the best option.  Without deprecation people will keep using cElementTree and we will have to keep it around forever (or at least until Python4 and then have a 3to4 to fix the import).
This might be fine, but as a developer I would still like Python to tell me "You can just import ElementTree now, there's no need to use cElementTree".  Maybe the deprecation can be added to 3.4?

P.S. I'm fine with keeping it around for several more versions, but if we eventually have to remove it, we would still have to warn the users beforehand.  The more we wait, the more users will be still using cElementTree by the time we will actually remove it.
msg153564 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-17 16:31
I don’t see benefits in removing cET.
msg160745 - (view) Author: Markus (cmn) Date: 2012-05-15 18:04
Hi,

the C implementation of ElementTree do not support namespaces for find/all/... .

To me this is a serious regression, as I rely on ElementTree namespace support, and 3.3 would break it with this change.
Breaking namespace support is a fundamental problem.

Please reconsider this therefore.

Code to reproduced attached - works fine with python 3.2.

As the C implementation of ElementTree and Element lack the namespace keyword for findall (and *all* the other methods), 
where namespaces are very important when dealing with xml, 
and it is not possible to prevent using the v implementation of ElementTree without changing the python install, 
I propose to revert this change.

Until the C implementation can do namespaces as well.
msg160746 - (view) Author: Markus (cmn) Date: 2012-05-15 18:14
The file was bad, sorry.
re-attached
msg160747 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2012-05-15 18:16
Markus (cmn): Please file a separate issue, which will be a release blocker for 3.3 release. (It's not the only regression.)
msg160748 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2012-05-15 18:19
Temporary very ugly workaround (before importing xml.etree.ElementTree) is:

import sys
sys.modules["_elementtree"] = None
msg160749 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-05-15 18:24
It seems to me that namespaces are actually supported, but they are accepted only as positional args and not keyword args, so this should be easy to fix.
msg160754 - (view) Author: Markus (cmn) Date: 2012-05-15 19:44
As advised I opened a new bug on this:
http://bugs.python.org/issue14818
msg161044 - (view) Author: Markus (cmn) Date: 2012-05-18 10:15
New bug - C implementation of ElementTree: Inheriting from Element breaks text member
http://bugs.python.org/issue14849
msg162843 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-06-15 04:49
Note: last traces of Python bootstrap code were removed from _elementtree in changeset 652d148bdc1d
History
Date User Action Args
2012-06-15 04:49:46eli.benderskysetmessages: + msg162843
2012-05-18 10:15:20cmnsetmessages: + msg161044
2012-05-15 19:44:56cmnsetmessages: + msg160754
2012-05-15 18:24:54ezio.melottisetmessages: + msg160749
2012-05-15 18:19:39Arfreversetmessages: + msg160748
2012-05-15 18:16:12Arfreversetmessages: + msg160747
2012-05-15 18:14:38cmnsetfiles: + findall_takes_no_keywords_anymore.py

messages: + msg160746
2012-05-15 18:13:53cmnsetfiles: - findall_takes_no_keywords_anymore.py
2012-05-15 18:04:23cmnsetfiles: + findall_takes_no_keywords_anymore.py

nosy: + cmn
messages: + msg160745

type: performance -> behavior
2012-02-17 16:31:13eric.araujosetmessages: + msg153564
2012-02-16 13:04:25ezio.melottisetmessages: + msg153480
2012-02-16 03:29:06eli.benderskysetmessages: + msg153453
2012-02-14 04:20:02eli.benderskysetmessages: - msg153327
2012-02-14 04:19:08eli.benderskysetmessages: + msg153327
2012-02-14 04:19:06eli.benderskysetmessages: + msg153326
2012-02-14 04:18:50eli.benderskysetmessages: - msg153325
2012-02-14 04:18:17eli.benderskysetmessages: + msg153325
2012-02-14 04:08:05pjenveysetnosy: + pjenvey
messages: + msg153323
2012-02-14 03:56:59eli.benderskysetmessages: + msg153321
2012-02-14 03:32:56eli.benderskysetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2012-02-14 03:31:22eli.benderskysetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg153319

stage: resolved -> needs patch
2012-02-13 12:34:52floxsetstatus: open -> closed
resolution: fixed
messages: + msg153265

stage: needs patch -> resolved
2012-02-13 11:15:18python-devsetmessages: + msg153259
2012-02-13 10:04:54python-devsetmessages: + msg153258
2012-02-13 09:18:00ezio.melottisetmessages: + msg153255
2012-02-13 04:25:12eli.benderskysetmessages: + msg153249
2012-02-12 21:07:45floxsetmessages: + msg153230
2012-02-12 19:11:14eli.benderskysetmessages: + msg153223
2012-02-12 19:06:22eli.benderskysetmessages: + msg153221
2012-02-12 12:52:26floxsetfiles: + issue13988_fold_cET_behind_ET_v2.diff

messages: + msg153204
2012-02-12 12:28:18floxsetmessages: + msg153203
2012-02-12 05:46:07eli.benderskysetmessages: + msg153180
2012-02-12 05:11:52eric.araujosetmessages: + msg153175
2012-02-12 04:35:55eli.benderskysetmessages: + msg153165
2012-02-12 04:21:17eli.benderskysetfiles: + issue13988_doc_news.1.patch

messages: + msg153160
2012-02-12 04:12:23eli.benderskysetmessages: + msg153158
2012-02-12 03:49:58eli.benderskysetmessages: + msg153155
2012-02-12 03:44:20eric.araujosetmessages: + msg153151
2012-02-11 19:29:26tshepangsetnosy: + tshepang
2012-02-11 11:59:42eli.benderskysetmessages: + msg153122
2012-02-11 10:39:17floxsetfiles: + issue13988_fold_cET_behind_ET.diff

messages: + msg153120
2012-02-11 10:29:55python-devsetnosy: + python-dev
messages: + msg153119
2012-02-11 09:37:29floxsetfiles: + issue13988_prepare_pep399_v2.diff
2012-02-11 09:35:22floxsetmessages: + msg153115
2012-02-11 09:23:48floxsetmessages: + msg153114
2012-02-11 08:58:00eli.benderskysetmessages: + msg153112
2012-02-11 08:39:14scodersetmessages: + msg153111
2012-02-11 08:08:56eli.benderskysetmessages: + msg153110
2012-02-11 07:41:27eli.benderskysetmessages: + msg153109
2012-02-11 05:32:23eric.araujosetmessages: + msg153101
2012-02-11 04:24:01Arfreversetnosy: + Arfrever
2012-02-11 04:05:59eli.benderskysetmessages: + msg153089
2012-02-11 03:34:51eli.benderskysetmessages: + msg153086
2012-02-10 21:39:51floxsetfiles: + issue13988_prepare_pep399.diff
keywords: + patch
messages: + msg153078
2012-02-10 20:43:29ezio.melottisetnosy: + eric.araujo
messages: + msg153075
2012-02-10 17:26:48eli.benderskysetmessages: + msg153068
2012-02-10 16:46:24ezio.melottisettype: performance
messages: + msg153066
stage: needs patch
2012-02-10 16:38:39eli.benderskysettype: performance -> (no value)
messages: + msg153065
stage: needs patch -> (no value)
2012-02-10 16:37:26eli.benderskysetmessages: + msg153064
2012-02-10 16:30:10floxsettype: performance
messages: + msg153063
stage: needs patch
2012-02-10 16:12:20eli.benderskysetmessages: + msg153062
2012-02-10 16:07:43ezio.melottisetnosy: + ezio.melotti
messages: + msg153061
2012-02-10 15:55:22scodersetmessages: + msg153059
2012-02-10 15:46:00eli.benderskysetmessages: + msg153058
2012-02-10 15:43:14eli.benderskysetmessages: + msg153057
2012-02-10 15:37:16scodersetmessages: + msg153056
2012-02-10 14:54:07eli.benderskysetmessages: + msg153053
2012-02-10 14:52:56eli.benderskycreate