New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CVE-2018-20406] memory exhaustion in Modules/_pickle.c:1393 #78837
Comments
python version: I found a bug in python pickle.load func. Can cause memory exhaustion DDOS. ./python pk.py poc cat ./pk.py |
[----------------------------------registers-----------------------------------] ..... |
>>> import pickletools
>>> pickletools.dis(b'\x80\x04\x95\x1d\x00\x00\x00\x00\x00\x00\x00}\x94(\x8c\x03age\x94K\x17\x8c\x03jobr\x8c\x07student\x94u.')
0: \x80 PROTO 4
2: \x95 FRAME 29
11: } EMPTY_DICT
12: \x94 MEMOIZE (as 0)
13: ( MARK
14: \x8c SHORT_BINUNICODE 'age'
19: \x94 MEMOIZE (as 1)
20: K BININT1 23
22: \x8c SHORT_BINUNICODE 'job'
27: r LONG_BINPUT 1953695628
32: u SETITEMS (MARK at 13)
33: d DICT no MARK exists on stack
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/serhiy/py/cpython/Lib/pickletools.py", line 2457, in dis
raise ValueError(errormsg)
ValueError: no MARK exists on stack Ignore the error of unbalanced MARK. The problem code is LONG_BINPUT with the excessive large argument 1953695628. The C implementation of pickle tries to resize the the memo list to the size twice larger than this index. And here an integer overflow occurred. This unlikely occurred in real world. The pickle needs to have more than 2**30-1 ≈ 10**9 memoized items for encountering this bug. It means that its size on disk and in memory should be tens or hundreds of gigabytes. Pickle is not the best format for serializing such amount of data. |
Should this go to 3.4 and 3.5 as well, since it is a security thing? http://people.canonical.com/~ubuntu-security/cve/2018/CVE-2018-20406.html |
I am not sure this issue should be classified as a security issue. It can cause DDOS, because pickle should not be used with untrusted data. If it is used, the program has more severe security issues than just DDOS. The crash could be triggered by accident, but this is very unlikely. I doubts that this happened even once in real world. Libraries used for handling a large amount of data (like NumPy) use more efficient pickle representation, and can provide even more efficient alternate serialization methods. Note that integers and floats are not memoized, this increases the complexity and size of data that could be affected by this bug. But I think that this fix needs a news entry. Do you mind to add it Benjamin? |
Does it even make sense to make a security patch for 2.7 for this one? |
Python 2.7 is not affected:
I reopen the issue because the issue should be fixed in 3.4 and 3.5 as well, since it has been marked as a vulnerability (it got a CVE number). |
And Modules/cPickle.c is that drastically different? |
Stupid me. I was surprised that Python 2.7 had no C accelerator. I was looking for Modules/pickle.c on my case sensitive Linux filesystem... |
New changeset a4ae828 by Benjamin Peterson in branch 'master': It seems like this patch changes the implementation of the internal "memo" object which is a custom C type in Python 3. In Python 2 cPickle, the memo is a regular dictionary and so I'm not sure that Python 2 is affected by this vulnerability. Can someone please confirm? |
Python 3.4 doesn't allow C99 constructs, so I had to update the patch to reorder iterator declarations. Just if any future colleague Python Linux distro maintainer needs it. |
In python2, Picklertype donot has tp init which has bug in python3 Pickler_Type. |
As I wrote in my previous comment, I don't think that Python 2.7 is affected by this issue. |
It's not 100% clear to me but based off this thread, it seems that this issue has been resolved. However, I'm still seeing similar behavior where using For example, What's interesting is sometimes it throws a MemoryError, but most of the time it just kills the process after waiting a while. I tested this on Python3.11 and Python3.12.1. Here's an example of this inconsistent behavior that I'm talking about. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: