[security] CVE-2017-1000158: Unsafe arithmetic in PyString_DecodeEscape #74842
Comments
|
In Python 2.7, there is a possible integer overflow in The relevant parts of the code are highlighted below: PyObject *PyString_DecodeEscape(const char *s,
Py_ssize_t len,
const char *errors,
Py_ssize_t unicode,
const char *recode_encoding)
{
int c;
char *p, *buf;
const char *end;
PyObject *v;
(1) Py_ssize_t newlen = recode_encoding ? 4*len:len;
(2) v = PyString_FromStringAndSize((char *)NULL, newlen);
if (v == NULL)
return NULL;
(3) p = buf = PyString_AsString(v);
end = s + len;
while (s < end) {
if (*s != '\\') {
non_esc:
#ifdef Py_USING_UNICODE
[...]
#else
*p++ = *s++;
#endif
continue;
[...]
}
}
(4) if (p-buf < newlen)
_PyString_Resize(&v, p - buf); /* v is cleared on error */
return v;
failed:
Py_DECREF(v);
return NULL;
}(1) If recode_encoding is true (i.e., non-null), we have an integer In the highly unlikely but definitely possible situation that we pass The attached file (poc-gen.py) produces a poc.py file which satisfies Note: To see the vulnerability in action, it is necessary to have an Proposed fix: Confirm that the multiplication will not overflow, |
jaybosamiya
mannequin
added
interpreter-core
|
Thank you for your report Jay. Even if it very unlikely that this can occurred unintentionally or be used for attack, this still is a bug and should be fixed. Do you want to provide a patch? |
|
I've made a patch that should fix the vulnerability. Please do let me know if changes are required. Thanks a lot :) PS: For anyone who looks at this later on, in my original message describing the issue, the line |
|
I couldn't reproduce using the poc, but seems python3.5 is also vulnerable to this bug. The code from py3.5 are quite similar to 2.7. |
|
Right, but it is not easy to exploit this bug. You need to parse Python sources longer than 512 MiB in 32-bit Python. Python 3.5 currently takes only fixes for security bugs. I left on to Larry to decide whether it is worth to port the fix to 3.5. |
|
I would welcome a backport of this for 3.5 and even 3.4 (if it's vulnerable, which it probably is). |
|
Python 3.4 also has the similar code as 3.5, but applying the same patch tests for it results in test errors: +======================================================================
+ERROR: test_anydbm_creation (test.test_dbm.TestCase-dbm.ndbm)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File "/<<PKGBUILDDIR>>/Lib/test/test_dbm.py", line 74, in test_anydbm_creation^M
+ self.read_helper(f)^M
+ File "/<<PKGBUILDDIR>>/Lib/test/test_dbm.py", line 115, in read_helper
+ self.assertEqual(self._dict[key], f[key.encode("ascii")])
+KeyError: b'0'
+
+======================================================================
+ERROR: test_anydbm_modification (test.test_dbm.TestCase-dbm.ndbm)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File "/<<PKGBUILDDIR>>/Lib/test/test_dbm.py", line 89, in test_anydbm_modification
+ self.read_helper(f)
+ File "/<<PKGBUILDDIR>>/Lib/test/test_dbm.py", line 115, in read_helper
+ self.assertEqual(self._dict[key], f[key.encode("ascii")])
+KeyError: b'0'
+
+======================================================================
+ERROR: test_anydbm_read (test.test_dbm.TestCase-dbm.ndbm)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File "/<<PKGBUILDDIR>>/Lib/test/test_dbm.py", line 95, in test_anydbm_read
+ self.read_helper(f)
+ File "/<<PKGBUILDDIR>>/Lib/test/test_dbm.py", line 115, in read_helper
+ self.assertEqual(self._dict[key], f[key.encode("ascii")])
+KeyError: b'0' |
|
I re-did the build here for python3.4 and couldn't reach the same test fail. So I'm assuming it was a false alarm. |
|
Leo kirotawa silva: "I re-did the build here for python3.4 and couldn't reach the same test fail. So I'm assuming it was a false alarm." Python 3.4 and 3.5 seem to be also vulnerable: PyObject *PyBytes_DecodeEscape(const char *s,
Py_ssize_t len,
const char *errors,
Py_ssize_t unicode,
const char *recode_encoding)
{
...
Py_ssize_t newlen = recode_encoding ? 4*len:len;
v = PyBytes_FromStringAndSize((char *)NULL, newlen);I don't think that Python 3.6 and 3.7 are vulnerable, the code was rewritten with the _PyBytesWriter API. The code got a new _PyBytes_DecodeEscapeRecode() helper function which calls _PyBytesWriter_WriteBytes(), and this function detects properly integer overflows. |
vstinner
changed the title
vstinner
changed the title
|
I added this vulnerability to the python-security website: |
|
I don't think it is worth to add this vulnerability to the python-security website. You need to compile a 1 GiB Python file on 32-bit system for reproducing it. It is very unlikely that this can happen by accident, and it is hard to used it in security attack. If you can make the attacked program compiling a 1 GiB Python file, you perhaps have easier ways to make a harm. |
|
Serhiy: "I don't think it is worth to add this vulnerability to the python-security website. You need to compile a 1 GiB Python file on 32-bit system for reproducing it. It is very unlikely that this can happen by accident, and it is hard to used it in security attack. If you can make the attacked program compiling a 1 GiB Python file, you perhaps have easier ways to make a harm." I'm trying to keep track of all CVEs. People are scared by CVE numbers :-( But it seems like any bug can get a CVE number, without any real evalution of the severity of the bug. I completed the description on python-security with your paragraph. FYI I wrote python-security to make sure that vulnerabilities are fixed in supported Python branches. Here it seems like we forgot to fix Python 3.4 and 3.5. |
|
Resetting to "needs patch", because we still need PRs for 3.4 and 3.5 (please!). |
|
New changeset fd8614c by larryhastings (Miro Hrončok) in branch '3.5': |
|
New changeset 6c004b4 by larryhastings (Miro Hrončok) in branch '3.4': |
|
Merged into 3.4 and 3.5. Thanks for the patches! Since I see 2.7 has already had the fix committed, and these are the only three versions affected, I'm marking as closed / resolved / fixed. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: