classification
Title: [security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP
Type: security Stage: resolved
Components: Tests Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: The Compiler, miss-islington, ned.deily, pablogsal, serhiy.storchaka, vstinner, zach.ware
Priority: normal Keywords: patch, security_issue

Created on 2020-10-05 14:40 by serhiy.storchaka, last changed 2020-11-04 13:09 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22566 merged serhiy.storchaka, 2020-10-05 14:49
PR 22575 merged The Compiler, 2020-10-06 10:45
PR 22576 merged miss-islington, 2020-10-06 12:15
PR 22577 merged miss-islington, 2020-10-06 12:15
PR 22578 merged miss-islington, 2020-10-06 12:38
PR 22579 merged miss-islington, 2020-10-06 12:38
Messages (19)
msg378036 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-05 14:40
As was reported by Florian Bruhin, Python testsuite calls eval() on content received via HTTP (in Lib/test/multibytecodec_support.py).
msg378104 - (view) Author: Florian Bruhin (The Compiler) * Date: 2020-10-06 09:15
I wonder if I should request a CVE for this as well? Just to make sure the word gets out to distributions/organizations/etc. running the Python testsuite, given that we can't be sure it which contexts this happens (and as it could be exploited by e.g. spoofing a WiFi network or so).
msg378105 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-06 09:24
I don't think that a CVE is justified.

I don't know anyone running the Python test suite on production. Only developers of Python itself run Python.
msg378106 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-06 09:25
Oops: Only developers of Python itself run the Python test suite.
msg378107 - (view) Author: Florian Bruhin (The Compiler) * Date: 2020-10-06 09:42
That assumption is false. For starters, distribution packagers do:

https://github.com/archlinux/svntogit-packages/blob/3fc85177e35d1ff9ab000950c5d1af9567730434/trunk/PKGBUILD#L72-L84

https://src.fedoraproject.org/rpms/python3.9/blob/master/f/python3.9.spec#_1168

When I build a Python from source (via an Arch User Repository package), I do as well, and so does anyone installing those packages by default.

Anyone building with --enable-optimizations (PGO) will likely do so as well, though I'm not sure if that runs this part of the testsuite.
msg378108 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-06 10:28
I'm not saying that this issue is not a vulnerability, just that the scope is limited.

By default, downloaded from the Internet are disabled. You have to opt-in for that using -u network (or -u all which enables the network resource) command line option of "./python -m test".

Impacted:

* "make testall", "make testuniversal" and "make buildbottest" commands are impacted (pass -u all to the test suite).

* Python buildbot workers are impacted: they run the "make buildbottest" command.

* Travis CI is impacted: it runs "./python -m test -uall,-cpu (...)".

* Multiple GitHub Action jobs are impacted (coverage, Windows, macOS, Ubuntu): run "-uall,-cpu".

* Azure Pipelines jobs are impacted: use -uall,-cpu.


> https://src.fedoraproject.org/rpms/python3.9/blob/master/f/python3.9.spec#_1168

Fedora packages are not impacted: no -u option is passed to the test suite.


> Anyone building with --enable-optimizations (PGO) will likely do so as well, though I'm not sure if that runs this part of the testsuite.

PGO build is not impacted, it uses "./python -m test --pgo" (download is disabled). Moreover, multibyte codec checks are not run by this command (see Lib/test/libregrtest/pgo.py, only test_codecs of codec tests is run).
msg378110 - (view) Author: Florian Bruhin (The Compiler) * Date: 2020-10-06 10:47
Thanks for the clarification - I wasn't aware those tests aren't run by default.

FWIW I found another place where a similar thing is done, though by chance it's probably not exploitable - see GH-22575.
msg378111 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-06 10:55
> FWIW I found another place where a similar thing is done, though by chance it's probably not exploitable - see GH-22575.

I agree that test_ucn is not exploitable, but it would be nice to harden it anyway.

Extract of the code:

            self.assertEqual(unicodedata.lookup(seqname), codepoints)
            with self.assertRaises(SyntaxError):
                self.checkletter(seqname, None)

test_ucn downloads http://www.pythontest.net/unicode/13.0.0/NamedSequences.txt and calls checkletter() on each line, but first it ensures that unicodedata.lookup(seqname) works as expected.

I don't see how it would be possible to inject arbitrary Python code in the 'seqname' variable without making unicodedata.lookup() to fail.
msg378114 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-06 11:07
I'm now tracking this vulnerability at:
https://python-security.readthedocs.io/vuln/cjk-codec-download-eval.html
msg378117 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-06 12:14
New changeset 2ef5caa58febc8968e670e39e3d37cf8eef3cab8 by Serhiy Storchaka in branch 'master':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566)
https://github.com/python/cpython/commit/2ef5caa58febc8968e670e39e3d37cf8eef3cab8
msg378118 - (view) Author: miss-islington (miss-islington) Date: 2020-10-06 12:37
New changeset b664a1df4ee71d3760ab937653b10997081b1794 by Miss Skeleton (bot) in branch '3.9':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566)
https://github.com/python/cpython/commit/b664a1df4ee71d3760ab937653b10997081b1794
msg378119 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-06 12:38
New changeset 6c6c256df3636ff6f6136820afaefa5a10a3ac33 by Miss Skeleton (bot) in branch '3.8':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22577)
https://github.com/python/cpython/commit/6c6c256df3636ff6f6136820afaefa5a10a3ac33
msg378120 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-06 12:39
Since it's a security vulnerability, I created backports to 3.6 and 3.7 as well.
msg378125 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-06 14:22
New changeset a8bf44d04915f7366d9f8dfbf84822ac37a4bab3 by Florian Bruhin in branch 'master':
bpo-41944: No longer call eval() on content received via HTTP in the UnicodeNames tests (GH-22575)
https://github.com/python/cpython/commit/a8bf44d04915f7366d9f8dfbf84822ac37a4bab3
msg379082 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-10-20 02:38
New changeset 43e523103886af66d6c27cd72431b5d9d14cd2a9 by Miss Skeleton (bot) in branch '3.7':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22578)
https://github.com/python/cpython/commit/43e523103886af66d6c27cd72431b5d9d14cd2a9
msg379085 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-10-20 04:46
New changeset e912e945f2960029d039d3390ea08835ad39374b by Miss Skeleton (bot) in branch '3.6':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22579)
https://github.com/python/cpython/commit/e912e945f2960029d039d3390ea08835ad39374b
msg379713 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-27 02:25
Thanks for the fix Serhiy and thanks Florian Bruhin for the bug report!
msg380319 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-11-04 13:09
The CVE-2020-27619 has been assigned to this issue.
msg380320 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-11-04 13:09
Red Hat advisory: https://access.redhat.com/security/cve/CVE-2020-27619
History
Date User Action Args
2020-11-04 13:09:52vstinnersetmessages: + msg380320
2020-11-04 13:09:32vstinnersetmessages: + msg380319
title: [security] Python testsuite calls eval() on content received via HTTP -> [security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP
2020-10-27 02:25:42vstinnersetmessages: + msg379713
2020-10-20 04:47:52ned.deilysetkeywords: + security_issue
status: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.6, Python 3.7
2020-10-20 04:46:17ned.deilysetmessages: + msg379085
2020-10-20 02:38:43ned.deilysetnosy: + ned.deily
messages: + msg379082
2020-10-06 14:22:06vstinnersetmessages: + msg378125
2020-10-06 12:39:20vstinnersetmessages: + msg378120
2020-10-06 12:38:57vstinnersetmessages: + msg378119
2020-10-06 12:38:19miss-islingtonsetpull_requests: + pull_request21574
2020-10-06 12:38:07miss-islingtonsetpull_requests: + pull_request21573
2020-10-06 12:37:44miss-islingtonsetmessages: + msg378118
2020-10-06 12:15:21miss-islingtonsetpull_requests: + pull_request21572
2020-10-06 12:15:11miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request21571
2020-10-06 12:14:59serhiy.storchakasetmessages: + msg378117
2020-10-06 11:07:40vstinnersetmessages: + msg378114
2020-10-06 10:55:47vstinnersetmessages: + msg378111
2020-10-06 10:47:19The Compilersetmessages: + msg378110
2020-10-06 10:45:56The Compilersetpull_requests: + pull_request21570
2020-10-06 10:28:20vstinnersetnosy: + zach.ware, pablogsal
messages: + msg378108
2020-10-06 09:42:03The Compilersetmessages: + msg378107
2020-10-06 09:25:14vstinnersetmessages: + msg378106
2020-10-06 09:24:54vstinnersetmessages: + msg378105
title: Python testsuite calls eval() on content received via HTTP -> [security] Python testsuite calls eval() on content received via HTTP
2020-10-06 09:15:43The Compilersetmessages: + msg378104
2020-10-05 14:49:59serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request21561
2020-10-05 14:40:52serhiy.storchakacreate