Title: marshal load bypass code.__new__ audit event
Type: security Stage: resolved
Components: Versions: Python 3.10, Python 3.9, Python 3.8
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, steve.dower, tkmk
Priority: normal Keywords: easy (C), patch

Created on 2020-07-01 06:58 by tkmk, last changed 2021-03-05 17:22 by tkmk. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 21271 merged tkmk, 2020-07-02 03:24
PR 21300 merged miss-islington, 2020-07-03 20:56
PR 21301 merged miss-islington, 2020-07-03 20:57
Messages (8)
msg372733 - (view) Author: Yunfan Zhan (tkmk) * Date: 2020-07-01 06:58
While `code.__new__` is being audited, using `marshal.loads` to create a code object will trigger no events. Therefore, either `marshal.load(s)` event itself should be audited, or `code.__new__` should be triggered when marshal type is TYPE_CODE.

Considering that importing from a pyc file also relys on unmarshalling code objects, and they have already been audited as `import`, I'm also wondering if auditing twice should be avoided for performance.
msg372771 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-07-01 17:06
I like using the existing event for unmarshalling code objects, assuming we have all the arguments available.

I'm not sure whether it's worth auditing all marshal.load() calls (just as we don't audit all pickle.load() calls). But depending on the code paths we may not have a choice.

Auditing the .pyc import twice (or more) is basically unavoidable, but it's not terrible anyway. When no hooks are set, it's negligible impact, and when someone adds a hook they are in control of how much work it does. Providing both events means they can choose to only handle one and still get the coverage.

Marking this as easy (C), since the hardest part is just going to be finding the right place to add the PySys_Audit call.
msg372876 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-07-02 19:51
Actually, a quick search of codeobject.c and a look at tkmk's PR makes it seem like the audit event should be being raised from inside PyCode_NewWithPosOnlyArgs anyway (which IIRC didn't exist when I first added the event, though it was probably there before it was merged).

In general, events should be raised after parameters have been validated, but before any work is done. And when all the other calls feed through a single function, just auditing that one function is enough.
msg372900 - (view) Author: Yunfan Zhan (tkmk) * Date: 2020-07-03 03:08
Before this, we only audit code.__new__ and code.replace, as these methods allow constructing arbitrary code objects, and we don't audit code object coming from the normal way (like compile,exec,eval).
If the event is raised in PyCode_NewWithPosOnlyArgs, is it ok that the compiled code is also audited?
msg372956 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-07-03 20:55
Ah, you're right. Thanks for double checking me :)

I'll merge the PR and do the backports. Thanks!
msg372957 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-07-03 20:56
New changeset d160e0f8e283d0a8737644588b38e8c6a07c134f by tkmikan in branch 'master':
bpo-41180: Audit code.__new__ when unmarshalling (GH-21271)
msg372959 - (view) Author: miss-islington (miss-islington) Date: 2020-07-03 21:13
New changeset c1d916595eb6979d4d87cc3e5216e26b3c6fac25 by Miss Islington (bot) in branch '3.8':
bpo-41180: Audit code.__new__ when unmarshalling (GH-21271)
msg372960 - (view) Author: miss-islington (miss-islington) Date: 2020-07-03 21:16
New changeset 1c776541a8805576c0a4363ca28c1d29423f02f6 by Miss Islington (bot) in branch '3.9':
bpo-41180: Audit code.__new__ when unmarshalling (GH-21271)
Date User Action Args
2021-03-05 17:22:18tkmksetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-07-03 21:16:26miss-islingtonsetmessages: + msg372960
2020-07-03 21:13:36miss-islingtonsetmessages: + msg372959
2020-07-03 20:57:13miss-islingtonsetpull_requests: + pull_request20450
2020-07-03 20:56:49miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request20449
2020-07-03 20:56:37steve.dowersetmessages: + msg372957
2020-07-03 20:55:04steve.dowersetmessages: + msg372956
2020-07-03 03:08:53tkmksetmessages: + msg372900
2020-07-02 19:51:16steve.dowersetmessages: + msg372876
2020-07-02 03:24:08tkmksetkeywords: + patch
stage: patch review
pull_requests: + pull_request20421
2020-07-01 17:06:00steve.dowersetkeywords: + easy (C)

messages: + msg372771
2020-07-01 06:58:32tkmkcreate