This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Update vendored copy to expat 2.4.1
Type: security Stage: resolved
Components: Extension Modules Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: lukasz.langa, miss-islington, ned.deily, pablogsal, scoder, sping, vstinner
Priority: normal Keywords: patch

Created on 2021-06-11 14:14 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
cpython_rebuild_expat_dir.sh vstinner, 2021-06-29 00:57
Pull Requests
URL Status Linked Edit
PR 26945 merged vstinner, 2021-06-29 00:56
PR 28031 merged miss-islington, 2021-08-29 14:08
PR 28032 merged miss-islington, 2021-08-29 14:08
PR 28033 merged miss-islington, 2021-08-29 14:08
PR 28042 merged lukasz.langa, 2021-08-29 15:17
PR 28080 merged ned.deily, 2021-08-31 06:34
PR 28617 merged pablogsal, 2021-09-29 12:22
PR 28619 merged miss-islington, 2021-09-29 13:18
PR 28620 merged miss-islington, 2021-09-29 13:18
PR 28621 merged miss-islington, 2021-09-29 13:19
PR 28624 merged lukasz.langa, 2021-09-29 13:53
PR 28627 merged miss-islington, 2021-09-29 14:30
Messages (20)
msg395634 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-11 14:14
Our vendored copy of Modules/expat/ should be updated to Expat 2.4.1 to retrieve the fix for the security vulnerabily CVE-2013-0340 "Billion Laughs":
https://blog.hartwork.org/posts/cve-2013-0340-billion-laughs-fixed-in-expat-2-4-0/

The table of vulnerabilities in Python XML parsers should be updated as well:
https://docs.python.org/dev/library/xml.html#xml-vulnerabilities

My outdated notes on Modules/expat/: copy of libexpat

* ./configure --with-system-expat
* Rationale: https://mail.python.org/pipermail/python-dev/2017-June/148287.html
* Used on Windows and macOS, Linux distributions use system libexpat
* Version: search for XML_MAJOR_VERSION in Modules/expat/expat.h
* Script to update it: see attached script to https://bugs.python.org/issue30947
* Recent update: https://bugs.python.org/issue30947
* Python 2.7, 3.3-3.6 use libexpat 2.2.1

https://pythondev.readthedocs.io/files.html
msg395642 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-06-11 15:46
(From PSRT list, Sebastian:)

 Please note that the vulnerability fix also added two new functions to
the API that would be great to have xml.parsers.expat expose to the
users for full control.  These are:

- XML_SetBillionLaughsAttackProtectionMaximumAmplification and
- XML_SetBillionLaughsAttackProtectionActivationThreshold

Module xml.parsers.expat.errors and its docs also needs 6 new error code
entries to be complete:

  /* Added in 2.0. */
  38 XML_ERROR_RESERVED_PREFIX_XML
  39 XML_ERROR_RESERVED_PREFIX_XMLNS
  40 XML_ERROR_RESERVED_NAMESPACE_URI

  /* Added in 2.2.1. */
  41 XML_ERROR_INVALID_ARGUMENT

  /* Added in 2.3.0. */
  42 XML_ERROR_NO_BUFFER

  /* Added in 2.4.0. */
  43 XML_ERROR_AMPLIFICATION_LIMIT_BREACH

With regard to the table of vulnerabilities mentioned in the ticket,
please note that vulnerability "quadratic blowup" is also fixed by
>=2.4.0.  Personally, I consider it a flavor of Billion Laughs and all
know variations are covered, including that one.
msg395649 - (view) Author: (sping) * Date: 2021-06-11 17:03
FTR that^^ Sebastian is me :)
msg396688 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-29 00:57
Attached cpython_rebuild_expat_dir.sh script updates Modules/expat/ to our libexpat copy to 2.4.1. I used it to create attached PR 26945.
msg400534 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-29 14:08
New changeset 3fc5d84046ddbd66abac5b598956ea34605a4e5d by Victor Stinner in branch 'main':
bpo-44394: Update libexpat copy to 2.4.1 (GH-26945)
https://github.com/python/cpython/commit/3fc5d84046ddbd66abac5b598956ea34605a4e5d
msg400537 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-29 14:31
New changeset c9c2a0bc9820f93f1020f3498f6893a3544c9b76 by Miss Islington (bot) in branch '3.8':
bpo-44394: Update libexpat copy to 2.4.1 (GH-26945) (GH-28033)
https://github.com/python/cpython/commit/c9c2a0bc9820f93f1020f3498f6893a3544c9b76
msg400538 - (view) Author: miss-islington (miss-islington) Date: 2021-08-29 14:32
New changeset 270678564c16452614a8acd93763bdf64fb4d286 by Miss Islington (bot) in branch '3.10':
bpo-44394: Update libexpat copy to 2.4.1 (GH-26945)
https://github.com/python/cpython/commit/270678564c16452614a8acd93763bdf64fb4d286
msg400539 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-29 14:36
New changeset 007221a43e566db08c0c5c00756d80dfd9dccafe by Miss Islington (bot) in branch '3.9':
bpo-44394: Update libexpat copy to 2.4.1 (GH-26945) (GH-28032)
https://github.com/python/cpython/commit/007221a43e566db08c0c5c00756d80dfd9dccafe
msg400547 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-29 15:24
3.6 will need a separate backport because it's using expat 2.2.6 at the moment (from b2260e59ff1eaf20de4738099005ddf507b7b27d).

3.7 conflicted since it didn't include local changes to the vendored 2.2.8 that were introduced in 3.8+. I fixed that, the backport is up.
msg400601 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-08-30 13:39
I created https://python-security.readthedocs.io/vuln/expat-billion-laughs.html to track this vulnerability.
msg400691 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-08-31 05:12
New changeset 79101b890ee021a901a8b6837a3a320d57adb725 by Łukasz Langa in branch '3.7':
[3.7] bpo-44394: Update libexpat copy to 2.4.1 (GH-26945) (GH-28042)
https://github.com/python/cpython/commit/79101b890ee021a901a8b6837a3a320d57adb725
msg400694 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-08-31 06:35
New changeset 910886a6448e4bf1edf49eeace4aa240b6403772 by Ned Deily in branch '3.6':
[3.6] bpo-44394: Update libexpat copy to 2.4.1 (GH-26945) (GH-28042) (GH-28080)
https://github.com/python/cpython/commit/910886a6448e4bf1edf49eeace4aa240b6403772
msg400695 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-08-31 07:05
PRs merged in 3.7 branch for release in 3.7.12 and in 3.6 branch for release in 3.6.15.
msg402783 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-09-28 15:43
The backport to 3.8 broke 3.8.12 in AIX:


0/Modules/_decimal/libmpdec/sixstep.o build/temp.aix-7.1-3.8/tmp/python3.8-3.8.12-0/Modules/_decimal/libmpdec/transpose.o -L. -L/opt/bb/lib -L/opt/bb/lib64 -R/opt/bb/lib64 -lm -o build/lib.aix-7.1-3.8/_decimal.cpython-38.so

*** WARNING: renaming "pyexpat" since importing it failed: rtld: 0712-001 Symbol _isnanf was referenced
from module build/lib.aix-7.1-3.8/pyexpat.cpython-38.so(), but a runtime definition of the symbol was not found.
msg402797 - (view) Author: (sping) * Date: 2021-09-28 17:56
For the AIX link error that Pablo brought up, there is merged pull request https://github.com/libexpat/libexpat/pull/510 upstream.
msg402869 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-09-29 13:18
New changeset 6c1154b9de29e1c9cd3d05f5289543e5cff73895 by Pablo Galindo Salgado in branch 'main':
bpo-44394: Ensure libexpat is linked against libm (GH-28617)
https://github.com/python/cpython/commit/6c1154b9de29e1c9cd3d05f5289543e5cff73895
msg402872 - (view) Author: miss-islington (miss-islington) Date: 2021-09-29 13:48
New changeset fafa213870193cf79557588ae8f9a4af570fd6e3 by Miss Islington (bot) in branch '3.9':
bpo-44394: Ensure libexpat is linked against libm (GH-28617)
https://github.com/python/cpython/commit/fafa213870193cf79557588ae8f9a4af570fd6e3
msg402875 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-29 14:13
New changeset 412ae8ab10734b72384c969181919cc4eb154406 by Miss Islington (bot) in branch '3.10':
[3.10] bpo-44394: Ensure libexpat is linked against libm (GH-28617) (GH-28621)
https://github.com/python/cpython/commit/412ae8ab10734b72384c969181919cc4eb154406
msg402880 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-29 14:36
New changeset 90004fca1cc3c6e3c9b2c3faae5cb1b7d7711648 by Miss Islington (bot) in branch '3.8':
[3.8] bpo-44394: Ensure libexpat is linked against libm (GH-28617) (GH-28620)
https://github.com/python/cpython/commit/90004fca1cc3c6e3c9b2c3faae5cb1b7d7711648
msg409459 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2022-01-01 14:27
I'd like to ask for clarification regarding issue 45321, which adds the missing error constants to the `expat` module. I consider those new features – it seems inappropriate to add new module constants in the middle of a release series. However, in this ticket here, the libexpat version was updated all the way back to Py3.6, to solve a security issue.

Should we also backport the error constants then?
History
Date User Action Args
2022-04-11 14:59:46adminsetgithub: 88560
2022-01-01 17:11:10gvanrossumsetnosy: - gvanrossum
2022-01-01 14:27:08scodersetnosy: + scoder
messages: + msg409459
2021-09-29 14:36:03lukasz.langasetmessages: + msg402880
2021-09-29 14:30:13miss-islingtonsetpull_requests: + pull_request26997
2021-09-29 14:13:46lukasz.langasetmessages: + msg402875
2021-09-29 13:53:49lukasz.langasetpull_requests: + pull_request26996
2021-09-29 13:48:03miss-islingtonsetmessages: + msg402872
2021-09-29 13:19:33miss-islingtonsetpull_requests: + pull_request26994
2021-09-29 13:18:50miss-islingtonsetpull_requests: + pull_request26993
2021-09-29 13:18:49pablogsalsetmessages: + msg402869
2021-09-29 13:18:45miss-islingtonsetpull_requests: + pull_request26992
2021-09-29 12:22:37pablogsalsetpull_requests: + pull_request26988
2021-09-28 17:56:52spingsetmessages: + msg402797
2021-09-28 15:43:45pablogsalsetmessages: + msg402783
2021-08-31 07:05:24ned.deilysetstatus: open -> closed
resolution: fixed
messages: + msg400695

stage: patch review -> resolved
2021-08-31 06:35:39ned.deilysetmessages: + msg400694
2021-08-31 06:34:02ned.deilysetpull_requests: + pull_request26523
2021-08-31 05:12:01ned.deilysetmessages: + msg400691
2021-08-30 13:39:34vstinnersetmessages: + msg400601
2021-08-29 15:24:19lukasz.langasetmessages: + msg400547
2021-08-29 15:17:28lukasz.langasetpull_requests: + pull_request26487
2021-08-29 14:36:44lukasz.langasetmessages: + msg400539
2021-08-29 14:32:58miss-islingtonsetmessages: + msg400538
2021-08-29 14:31:35lukasz.langasetmessages: + msg400537
2021-08-29 14:08:47miss-islingtonsetpull_requests: + pull_request26478
2021-08-29 14:08:42miss-islingtonsetpull_requests: + pull_request26477
2021-08-29 14:08:37miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26476
2021-08-29 14:08:31lukasz.langasetmessages: + msg400534
2021-06-29 00:57:30vstinnersetfiles: + cpython_rebuild_expat_dir.sh

messages: + msg396688
2021-06-29 00:56:15vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request25512
2021-06-21 15:33:04vstinnersetnosy: + ned.deily, lukasz.langa, pablogsal
2021-06-11 17:15:01spingsettitle: [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Updated to vendoed copy to expat 2.4.1 -> [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Update vendored copy to expat 2.4.1
2021-06-11 17:03:52spingsetnosy: + sping
messages: + msg395649
2021-06-11 15:46:25gvanrossumsetnosy: + gvanrossum
messages: + msg395642
2021-06-11 14:14:07vstinnercreate