classification
Title: Update embeded copy of libexpat from 2.2.1 to 2.2.3
Type: security Stage:
Components: Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, gregory.p.smith, haypo, ned.deily
Priority: normal Keywords:

Created on 2017-07-17 14:18 by haypo, last changed 2017-08-16 14:38 by haypo.

Files
File name Uploaded Description Edit
cpython_rebuild_expat_dir.sh haypo, 2017-08-16 14:35
Pull Requests
URL Status Linked Edit
PR 3106 open haypo, 2017-08-16 14:34
Messages (7)
msg298525 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-07-17 14:18
libexpat released a new version 2.2.2 which seems to contain 2 or 3 security fixes. I'm not sure that Python is affected by these bugs.

https://github.com/libexpat/libexpat/blob/R_2_2_2/expat/Changes#L5

Release 2.2.2 Wed July 12 2017
        Security fixes:
             #43  Protect against compilation without any source of high
                    quality entropy enabled, e.g. with CMake build system;
                    commit ff0207e6076e9828e536b8d9cd45c9c92069b895
             #60  Windows with _UNICODE:
                    Unintended use of LoadLibraryW with a non-wide string
                    resulted in failure to load advapi32.dll and degradation
                    in quality of used entropy when compiled with _UNICODE for
                    Windows; you can launch existing binaries with
                    EXPAT_ENTROPY_DEBUG=1 in the environment to inspect the
                    quality of entropy used during runtime; commits
                    * 95b95032f907ef1cd17ee7a9a1768010a825d61d
                    * 73a5a2e9c081f49f2d775cf7ced864158b68dc80
   [MOX-006]      Fix non-NULL parser parameter validation in XML_Parse;
                    resulted in NULL dereference, previously;
                    commit ac256dafdffc9622ab0dc2c62fcecb0dfcfa71fe

        Bug fixes:
             #69  Fix improper use of unsigned long long integer literals

        Other changes:
             #73  Start requiring a C99 compiler
             #49  Fix "==" Bashism in configure script
             #50  Fix too eager getrandom detection for Debian GNU/kFreeBSD
             #52    and macOS
             #51  Address lack of stdint.h in Visual Studio 2003 to 2008
             #58  Address compile warnings
             #68  Fix "./buildconf.sh && ./configure" for some versions
                    of Dash for /bin/sh
             #72  CMake: Ease use of Expat in context of a parent project
                    with multipe CMakeLists.txt files
             #72  CMake: Resolve mistaken executable permissions
             #76  Address compile warning with -DNDEBUG (not recommended!)
             #77  Address compile warning about macro redefinition

        Special thanks to:
            Alexander Bluhm
            Ben Boeckel
            Cătălin Răceanu
            Kerin Millar
            László Böszörményi
            S. P. Zeidler
            Segev Finer
            Václav Slavík
            Victor Stinner
            Viktor Szakats
                 and
Radically Open Security

--

Previous issue for expat 2.2.1: issue #30694.
msg298528 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-07-17 14:24
> #51  Address lack of stdint.h in Visual Studio 2003 to 2008

FYI this change only impacts Python 2.7, since Python 3.3 and newer requires Visual Studio 2010 or newer, and I already backported (cherry-picked) this specific commit in Python 2.7:
https://github.com/python/cpython/pull/2312/commits

> #58  Address compile warnings

That's my small contribution, so coming from CPython :-)
https://github.com/libexpat/libexpat/pull/58

> #76  Address compile warning with -DNDEBUG (not recommended!)

Nice contributions from Segev Finer, coming from CPython ;-)
https://github.com/libexpat/libexpat/issues/76

> #77  Address compile warning about macro redefinition

Another contribution of Segev Finer, already fixed downstream (in Python):
https://github.com/libexpat/libexpat/pull/77
msg298529 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-07-17 14:28
About the 3 security fixes (is the last change a security fix?).

"""
             #43  Protect against compilation without any source of high
                    quality entropy enabled, e.g. with CMake build system;
                    commit ff0207e6076e9828e536b8d9cd45c9c92069b895
"""

Since Python uses its own entropy source, I don't think that this change impacts us.

https://github.com/libexpat/libexpat/commit/ff0207e6076e9828e536b8d9cd45c9c92069b895


"""
             #60  Windows with _UNICODE:
                    Unintended use of LoadLibraryW with a non-wide string
                    resulted in failure to load advapi32.dll and degradation
                    in quality of used entropy when compiled with _UNICODE for
                    Windows; you can launch existing binaries with
                    EXPAT_ENTROPY_DEBUG=1 in the environment to inspect the
                    quality of entropy used during runtime; commits
                    * 95b95032f907ef1cd17ee7a9a1768010a825d61d
                    * 73a5a2e9c081f49f2d775cf7ced864158b68dc80
"""

I don't understand the consequence of this specific bug.

https://github.com/libexpat/libexpat/commit/95b95032f907ef1cd17ee7a9a1768010a825d61d
https://github.com/libexpat/libexpat/commit/73a5a2e9c081f49f2d775cf7ced864158b68dc80


"""
   [MOX-006]      Fix non-NULL parser parameter validation in XML_Parse;
                    resulted in NULL dereference, previously;
                    commit ac256dafdffc9622ab0dc2c62fcecb0dfcfa71fe
"""

I'm not sure that it's possible to call XML_Parse() with NULL in Python.

https://github.com/libexpat/libexpat/commit/ac256dafdffc9622ab0dc2c62fcecb0dfcfa71fe
msg300365 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-08-16 14:21
Expat 2.2.3 was released:

Release 2.2.3 Wed August 2 2017
        Security fixes:
             #82  CVE-2017-11742 -- Windows: Fix DLL hijacking vulnerability
                    using Steve Holme's LoadLibrary wrapper for/of cURL

        Bug fixes:
             #85  Fix a dangling pointer issue related to realloc

        Other changes:
                  Increase code coverage
             #91  Linux: Allow getrandom to fail if nonblocking pool has not
                    yet been initialized and read /dev/urandom then, instead.
                    This is in line with what recent Python does.
             #81  Pre-10.7/Lion macOS: Support entropy from arc4random
             #86  Check that a UTF-16 encoding in an XML declaration has the
                    right endianness
        #4 #5 #7  Recover correctly when some reallocations fail
                  Repair "./configure && make" for systems without any
                    provider of high quality entropy
                    and try reading /dev/urandom on those
                  Ensure that user-defined character encodings have converter
                    functions when they are needed
                  Fix mis-leading description of argument -c in xmlwf.1
                  Rely on macro HAVE_ARC4RANDOM_BUF (rather than __CloudABI__)
                    for CloudABI
            #100  Fix use of SIPHASH_MAIN in siphash.h
             #23  Test suite: Fix memory leaks
                  Version info bumped from 7:4:6 to 7:5:6

        Special thanks to:
            Chanho Park
            Joe Orton
            Pascal Cuoq
            Rhodri James
            Simon McVittie
            Vadim Zeitlin
            Viktor Szakats
                 and
Core Infrastructure Initiative
msg300367 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-08-16 14:30
Previous update: bpo-30694.
msg300368 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-08-16 14:35
cpython_rebuild_expat_dir.sh: Script used to update Modules/expat/ to 2.2.3. The script now uses the libexpat Git repository. Previously, I used tarballs.
msg300369 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-08-16 14:38
> #82  CVE-2017-11742 -- Windows: Fix DLL hijacking vulnerability using Steve Holme's LoadLibrary wrapper for/of cURL

https://github.com/libexpat/libexpat/issues/82

I don't think that this bug affects Python since Python sets a hash secret.
History
Date User Action Args
2017-08-16 14:38:43hayposetmessages: + msg300369
2017-08-16 14:35:08hayposetfiles: + cpython_rebuild_expat_dir.sh

messages: + msg300368
2017-08-16 14:34:32hayposetpull_requests: + pull_request3145
2017-08-16 14:30:37hayposetmessages: + msg300367
2017-08-16 14:22:54hayposettitle: Update embeded copy of libexpat to 2.2.2 -> Update embeded copy of libexpat from 2.2.1 to 2.2.3
2017-08-16 14:21:19hayposetmessages: + msg300365
2017-07-17 14:28:24hayposetmessages: + msg298529
2017-07-17 14:24:45hayposetmessages: + msg298528
2017-07-17 14:18:15haypocreate