As pointed out in #14234, our embedded copy of expat used by pyexpat for xml parsing in Modules/expat/ is out of date.  There have been many fixes to expat that we have not applied including a few potential crash and security fixes.

We should upgrade it wholesale to the latest version for 3.3.

Someone should also audit expat changes to see if there are security fixes for expat that should be backported to 2.6/2.7/3.1/3.2 as platforms without a system expat such as Windows (and 2.6 and 3.1) will contain those problems.

I am marking this a release blocker for 3.3 to ensure expat is updated before then.  I would *not* hold up the existing round of release candidates for this, the next security+bugfix updates can contain these changes.

What would be awesome is if we could just kill embedded versions of libraries like expat.

Indeed!  How do we do that on windows and osx where these may not exist outside of Python?

We already require a set of external dependency libraries on windows, could we just add expat to the list?

> What would be awesome is if we could just kill embedded versions of libraries like expat.

It practically wouldn't change much, since we still bundle them as part
of Windows binaries.

It is much less effort for us to simply take a new version of an external library and recompile rather than consider it part of our code that shouldn't change within a release and manually deal with patching it and cherry picking patches onto it.

Greg, are you still going to update expat in time for 3.3?  Otherwise this doesn't block 3.3 anymore.

Deferring for beta1 at least.

Thanks. I still intend to get to this.
 On Jun 23, 2012 1:59 AM, "Georg Brandl" <report@bugs.python.org> wrote:

>
> Georg Brandl <georg@python.org> added the comment:
>
> Deferring for beta1 at least.
>
> ----------
> priority: release blocker -> deferred blocker
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue14340>
> _______________________________________
>

Moving back to blocker for beta2.

New changeset e4dc8be9a72f by Gregory P. Smith in branch 'default':
Update the embedded copy of the expat XML parser to 2.1.0.  It brings
http://hg.python.org/cpython/rev/e4dc8be9a72f

Updated in 3.3 for beta2.

I'll leave it up to release managers to decide if they want to apply these updates for future 2.7 and 3.2 releases (trivial, just be sure to keep our one local modification adding the "#define XML_HAS_SET_HASH_SALT" to expat.h when you do it).

I'm leaving it as a release blocker for 2.7.4 and 3.2.4 so that they see it.

Reasons why it is a good idea to apply this change to 2.7.4 and 3.2.4:

* Memory leak in poolGrow (CVE-2012-1148)
* Resource leak in readfilemap.c (CVE-2012-1147)
* Buffer over-read and crash in big2_toUtf8 (CVE-2009-3560)
* Parser crash with special UTF-8 sequences (CVE-2009-3270)
* Dangling positionPtr after error (2855609) - http://sourceforge.net/tracker/?func=detail&aid=2855609&group_id=10127&atid=110127 - Specifically reported by a pyexpat user.
* Unitialized memory returned from XML_Parse (3206497) - http://sourceforge.net/tracker/?func=detail&aid=3206497&group_id=10127&atid=110127

The features 2.1.0 adds over 2.0.x are not exposed to pyexpat or Python users.

If you could apply this to 2.7, that'd be great.

New changeset c73a1f96dd9b by Gregory P. Smith in branch '2.7':
Update the embedded copy of the expat XML parser to 2.1.0.  It brings
http://hg.python.org/cpython/rev/c73a1f96dd9b

Then I guess there is no reason not to put it in 3.2.4.

Then I guess there is no reason not to put it in 3.2.4.

Greg, if you are fine please apply to 3.2 or indicate if it is enough to apply the same patch as on 3.3/default.

New changeset d2f6f63e73af by Gregory P. Smith in branch '3.2':
Update the embedded copy of the expat XML parser to 2.1.0.  It brings
http://hg.python.org/cpython/rev/d2f6f63e73af

done.

btw, it looks like benjamin.peterson did it for 2.7 yesterday morning but when 'hg graft' is used to apply a change from another branch the roundup notification mentions the original commit's author, not the person who did the push of the graft.