This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: safely eval serialized dict/list data from arbitrary string over web with no side effects
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, georg.brandl, kaizhu, skip.montanaro
Priority: normal Keywords:

Created on 2011-01-20 03:29 by kaizhu, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (4)
msg126586 - (view) Author: kai zhu (kaizhu) Date: 2011-01-20 03:29
rather than serialize python dicts & list to json / xml / protocol buffer for web use, its more efficient to just serialize w/ repr() & then use eval(), if only there was a way to guarantee arbitrary code can't b executed.

this is a very simple proposed method for the latter.  b4 eval(), it compiles string to python code object & checks for:
1. co_names list can only contain 'False', 'None', 'True'
   -this ensures no function call can b made
2. co_consts list cannot contain code objects
   -embedded lambda's r forbidden.
3. grave accents are explicitly forbidden.

here is the code for both python2.5 (intended for google appengine) & python 3k:

## safe_eval.py
import sys, types

if sys.version_info[0] == 2: ## py2x
  _co_safe = 'co_argcount co_nlocals co_varnames co_filename co_freevars co_cellvars'.split(' ')
else: ## py3k
  _co_safe = 'co_argcount co_kwonlyargcount co_nlocals co_names co_varnames co_filename co_freevars co_cellvars'.split(' ')

## safely eval string with no side-effects
def safe_eval(ss):
  if not ss: return None
  if '`' in ss: raise ValueError('grave accent "`" forbidden')
  cc = compile(ss, '', 'eval')
  for aa in _co_safe:
    if getattr(cc, aa): raise ValueError(aa + ' must be empty / none / zero')
  for aa in cc.co_names:
    if aa not in ['False', 'None', 'True']: raise ValueError('co_names can only contain False, None, True')
  for aa in cc.co_consts:
    if isinstance(aa, types.CodeType): raise TypeError('code objects not allowed in co_consts')
  return eval(cc, {})



python2.5
Python 2.5.5 (r255:77872, Nov 28 2010, 19:00:19) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from safe_eval import safe_eval

>>> safe_eval('[False, None, True, {1:2}]')
[False, None, True, {1: 2}]

>>> safe_eval('[None, False, True, {1:2}, evil_code()]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "safe_eval.py", line 19, in safe_eval
    if aa not in ['False', 'None', 'True']: raise ValueError('co_names can only contain False, None, True')
ValueError: co_names can only contain False, None, True

>>> safe_eval('[None, False, True, {1:2}, `evil_code()`]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "safe_eval.py", line 14, in safe_eval
    if '`' in ss: raise ValueError('grave accent "`" forbidden')
ValueError: grave accent "`" forbidden

>>> safe_eval('[None, False, True, {1:2}, lambda: evil_code()]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "safe_eval.py", line 21, in safe_eval
    if isinstance(aa, types.CodeType): raise TypeError('code objects not allowed in co_consts')
TypeError: code objects not allowed in co_consts
msg126589 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-20 06:03
> rather than serialize python dicts & list to json ...
> its more efficient to just serialize w/ repr() & then use eval()

Do you have benchmarks that support this claim?
msg126591 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-01-20 06:19
Have a look at ast.literal_eval().
msg126638 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2011-01-20 20:03
If you intend this to be "safe" in the security sense of the word, I
suggest you release it in PyPI and post a note on comp.lang.python
(a.k.a. python-list@python.org) asking people to try and break it.
History
Date User Action Args
2022-04-11 14:57:11adminsetgithub: 55162
2011-01-20 20:03:18skip.montanarosetnosy: + skip.montanaro
messages: + msg126638
2011-01-20 06:19:11georg.brandlsetstatus: open -> closed

nosy: + georg.brandl
messages: + msg126591

resolution: works for me
2011-01-20 06:03:41belopolskysetnosy: + belopolsky

messages: + msg126589
versions: - Python 2.5
2011-01-20 03:29:31kaizhucreate