[security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP #86110

serhiy-storchaka · 2020-10-05T14:40:52Z

BPO	41944
Nosy	@vstinner, @ned-deily, @zware, @serhiy-storchaka, @The-Compiler, @pablogsal, @miss-islington
PRs	bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests #22566 bpo-41944: No longer call eval() on content received via HTTP in the UnicodeNames tests #22575 [3.9] bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) #22576 [3.8] bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) #22577 [3.7] bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) #22578 [3.6] bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) #22579

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-10-20.04:47:52.035>
created_at = <Date 2020-10-05.14:40:52.277>
labels = ['type-security', '3.8', '3.9', '3.10', '3.7', 'tests']
title = '[security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP'
updated_at = <Date 2020-11-04.13:09:52.449>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2020-11-04.13:09:52.449>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-20.04:47:52.035>
closer = 'ned.deily'
components = ['Tests']
creation = <Date 2020-10-05.14:40:52.277>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 41944
keywords = ['patch', 'security_issue']
message_count = 19.0
messages = ['378036', '378104', '378105', '378106', '378107', '378108', '378110', '378111', '378114', '378117', '378118', '378119', '378120', '378125', '379082', '379085', '379713', '380319', '380320']
nosy_count = 7.0
nosy_names = ['vstinner', 'ned.deily', 'zach.ware', 'serhiy.storchaka', 'The Compiler', 'pablogsal', 'miss-islington']
pr_nums = ['22566', '22575', '22576', '22577', '22578', '22579']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue41944'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

serhiy-storchaka · 2020-10-05T14:40:52Z

As was reported by Florian Bruhin, Python testsuite calls eval() on content received via HTTP (in Lib/test/multibytecodec_support.py).

The-Compiler · 2020-10-06T09:15:43Z

I wonder if I should request a CVE for this as well? Just to make sure the word gets out to distributions/organizations/etc. running the Python testsuite, given that we can't be sure it which contexts this happens (and as it could be exploited by e.g. spoofing a WiFi network or so).

vstinner · 2020-10-06T09:24:55Z

I don't think that a CVE is justified.

I don't know anyone running the Python test suite on production. Only developers of Python itself run Python.

vstinner · 2020-10-06T09:25:14Z

Oops: Only developers of Python itself run the Python test suite.

The-Compiler · 2020-10-06T09:42:03Z

That assumption is false. For starters, distribution packagers do:

https://github.com/archlinux/svntogit-packages/blob/3fc85177e35d1ff9ab000950c5d1af9567730434/trunk/PKGBUILD#L72-L84

https://src.fedoraproject.org/rpms/python3.9/blob/master/f/python3.9.spec#_1168

When I build a Python from source (via an Arch User Repository package), I do as well, and so does anyone installing those packages by default.

Anyone building with --enable-optimizations (PGO) will likely do so as well, though I'm not sure if that runs this part of the testsuite.

vstinner · 2020-10-06T10:28:20Z

I'm not saying that this issue is not a vulnerability, just that the scope is limited.

By default, downloaded from the Internet are disabled. You have to opt-in for that using -u network (or -u all which enables the network resource) command line option of "./python -m test".

Impacted:

"make testall", "make testuniversal" and "make buildbottest" commands are impacted (pass -u all to the test suite).
Python buildbot workers are impacted: they run the "make buildbottest" command.
Travis CI is impacted: it runs "./python -m test -uall,-cpu (...)".
Multiple GitHub Action jobs are impacted (coverage, Windows, macOS, Ubuntu): run "-uall,-cpu".
Azure Pipelines jobs are impacted: use -uall,-cpu.

https://src.fedoraproject.org/rpms/python3.9/blob/master/f/python3.9.spec#_1168

Fedora packages are not impacted: no -u option is passed to the test suite.

Anyone building with --enable-optimizations (PGO) will likely do so as well, though I'm not sure if that runs this part of the testsuite.

PGO build is not impacted, it uses "./python -m test --pgo" (download is disabled). Moreover, multibyte codec checks are not run by this command (see Lib/test/libregrtest/pgo.py, only test_codecs of codec tests is run).

The-Compiler · 2020-10-06T10:47:20Z

Thanks for the clarification - I wasn't aware those tests aren't run by default.

FWIW I found another place where a similar thing is done, though by chance it's probably not exploitable - see #66765.

vstinner · 2020-10-06T10:55:48Z

FWIW I found another place where a similar thing is done, though by chance it's probably not exploitable - see #66765.

I agree that test_ucn is not exploitable, but it would be nice to harden it anyway.

Extract of the code:

        self.assertEqual(unicodedata.lookup(seqname), codepoints)
        with self.assertRaises(SyntaxError):
            self.checkletter(seqname, None)

test_ucn downloads http://www.pythontest.net/unicode/13.0.0/NamedSequences.txt and calls checkletter() on each line, but first it ensures that unicodedata.lookup(seqname) works as expected.

I don't see how it would be possible to inject arbitrary Python code in the 'seqname' variable without making unicodedata.lookup() to fail.

vstinner · 2020-10-06T11:07:40Z

I'm now tracking this vulnerability at:
https://python-security.readthedocs.io/vuln/cjk-codec-download-eval.html

serhiy-storchaka · 2020-10-06T12:14:59Z

New changeset 2ef5caa by Serhiy Storchaka in branch 'master':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566)
2ef5caa

miss-islington · 2020-10-06T12:37:44Z

New changeset b664a1d by Miss Skeleton (bot) in branch '3.9':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566)
b664a1d

vstinner · 2020-10-06T12:38:58Z

New changeset 6c6c256 by Miss Skeleton (bot) in branch '3.8':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22577)
6c6c256

vstinner · 2020-10-06T12:39:21Z

Since it's a security vulnerability, I created backports to 3.6 and 3.7 as well.

vstinner · 2020-10-06T14:22:06Z

New changeset a8bf44d by Florian Bruhin in branch 'master':
bpo-41944: No longer call eval() on content received via HTTP in the UnicodeNames tests (GH-22575)
a8bf44d

ned-deily · 2020-10-20T02:38:43Z

New changeset 43e5231 by Miss Skeleton (bot) in branch '3.7':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22578)
43e5231

ned-deily · 2020-10-20T04:46:18Z

New changeset e912e94 by Miss Skeleton (bot) in branch '3.6':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22579)
e912e94

vstinner · 2020-10-27T02:25:43Z

Thanks for the fix Serhiy and thanks Florian Bruhin for the bug report!

vstinner · 2020-11-04T13:09:32Z

The CVE-2020-27619 has been assigned to this issue.

vstinner · 2020-11-04T13:09:52Z

Red Hat advisory: https://access.redhat.com/security/cve/CVE-2020-27619

serhiy-storchaka · 2022-07-23T07:02:48Z

@gvanrossum, is there anything else that needs to be done in this issue?

gvanrossum · 2022-07-23T15:46:09Z

Whoops, didn’t mean to reopen. This has happened a few times — I look at an issue, maybe subscribe, and it gets reopened?

serhiy-storchaka added 3.10 only security fixes 3.8 only security fixes 3.9 only security fixes tests Tests in the Lib/test dir type-security A security issue labels Oct 5, 2020

vstinner changed the title ~~Python testsuite calls eval() on content received via HTTP~~ [security] Python testsuite calls eval() on content received via HTTP Oct 6, 2020

ned-deily added the 3.7 (EOL) end of life label Oct 20, 2020

ned-deily closed this as completed Oct 20, 2020

ned-deily added the 3.7 (EOL) end of life label Oct 20, 2020

ned-deily closed this as completed Oct 20, 2020

vstinner changed the title ~~[security] Python testsuite calls eval() on content received via HTTP~~ [security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP Nov 4, 2020

ezio-melotti transferred this issue from another repository Apr 10, 2022

gvanrossum reopened this Jul 22, 2022

gvanrossum closed this as completed Jul 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP #86110

[security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP #86110

serhiy-storchaka commented Oct 5, 2020

serhiy-storchaka commented Oct 5, 2020

The-Compiler mannequin commented Oct 6, 2020

vstinner commented Oct 6, 2020

vstinner commented Oct 6, 2020

The-Compiler mannequin commented Oct 6, 2020

vstinner commented Oct 6, 2020

The-Compiler mannequin commented Oct 6, 2020

vstinner commented Oct 6, 2020

vstinner commented Oct 6, 2020

serhiy-storchaka commented Oct 6, 2020

miss-islington commented Oct 6, 2020

vstinner commented Oct 6, 2020

vstinner commented Oct 6, 2020

vstinner commented Oct 6, 2020

ned-deily commented Oct 20, 2020

ned-deily commented Oct 20, 2020

vstinner commented Oct 27, 2020

vstinner commented Nov 4, 2020

vstinner commented Nov 4, 2020

serhiy-storchaka commented Jul 23, 2022

gvanrossum commented Jul 23, 2022

[security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP #86110

[security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP #86110

Comments

serhiy-storchaka commented Oct 5, 2020

serhiy-storchaka commented Oct 5, 2020

The-Compiler mannequin commented Oct 6, 2020

vstinner commented Oct 6, 2020

vstinner commented Oct 6, 2020

The-Compiler mannequin commented Oct 6, 2020

vstinner commented Oct 6, 2020

The-Compiler mannequin commented Oct 6, 2020

vstinner commented Oct 6, 2020

vstinner commented Oct 6, 2020

serhiy-storchaka commented Oct 6, 2020

miss-islington commented Oct 6, 2020

vstinner commented Oct 6, 2020

vstinner commented Oct 6, 2020

vstinner commented Oct 6, 2020

ned-deily commented Oct 20, 2020

ned-deily commented Oct 20, 2020

vstinner commented Oct 27, 2020

vstinner commented Nov 4, 2020

vstinner commented Nov 4, 2020

serhiy-storchaka commented Jul 23, 2022

gvanrossum commented Jul 23, 2022