This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ast.literal_eval() doesn't support empty sets
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: BTaskaya, docs@python, pablogsal, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2019-12-29 21:42 by rhettinger, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 17742 merged rhettinger, 2019-12-29 21:48
Messages (11)
msg359007 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-12-29 21:42
We already support sets but not empty sets.  After the PR, this now works:

    >>> from ast import literal_eval
    >>> literal_eval('set()')
    set()

If we wanted, it would be a simple matter to extend it frozensets:

    >>> literal_eval('frozenset({10, 20, 30})')
    frozenset({10, 20, 30})
msg359010 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-12-29 21:57
set() is neither literal nor container display.
msg359012 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-12-29 22:35
> set() is neither literal nor container display.

Yes, that is obvious.  However, we do support sets and set() is how make an empty set.   

It is weird to support sets but not empty sets, especially when it is so easy to do so safely.
msg359226 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-01-03 05:21
New changeset 4fcf5c12a37a8d3d8d6303c44c223dbc8d568cfd by Raymond Hettinger in branch 'master':
bpo-39158: ast.literal_eval() doesn't support empty sets (GH-17742)
https://github.com/python/cpython/commit/4fcf5c12a37a8d3d8d6303c44c223dbc8d568cfd
msg359251 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-01-03 19:32
The function literal_eval is not safe anymore as the constructor can be intercepted:

>>> import builtins
>>> def evil_code(*args):
...     print("Something evil")
...
>>> builtins.set = evil_code
>>> ast.literal_eval("set()")
Something evil


I think we should either use {0}.__class__.

Also, the documentation now is wrong as the function does more than evaluate literals or container displays.
msg359252 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-01-03 19:40
> The function literal_eval is not safe anymore as the constructor can be intercepted:

Well, actually it can also be done with any other builtin via the same trick, so this is no different but is only slightly more obvious that it can be done.

I still find a bit weird that the documentation says "literal or container display" but 'set' is neither.
msg359253 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-01-03 19:58
I am re-closing the issue as I don't think is worth complicating the docs for this edge case and the security concern is non existent.

Apologies for the noise. If someone feels strongly about the documentation, they can reopen the issue.
msg359287 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-01-04 13:41
The documentation for ast.literal_eval():

   Safely evaluate an expression node or a string containing a Python literal or
   container display.  The string or node provided may only consist of the
   following Python literal structures: strings, bytes, numbers, tuples, lists,
   dicts, sets, booleans, and ``None``.

https://docs.python.org/3/library/ast.html#ast.literal_eval

If we are going to add support of a function call set(), we should change the documentation.

And if add support of non-literals, where should we stop? Should we support also frozenset() and bytearray()? inf and nan? infj and nanj? complex()? Ellipsis? __debug__?
msg359292 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-01-04 15:02
> And if add support of non-literals, where should we stop? Should we support also frozenset() and bytearray()? inf and nan? infj and nanj? complex()? Ellipsis? __debug__?

Then the name of the function would be a bit misleading (for frozenset() and bytearray()), no? I think the set() change is somehow inconsistent already with the function name.
msg359453 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-01-06 19:24
> The function literal_eval is not safe anymore as the 
> constructor can be intercepted

"Safe" means safe from user input to literal_eval().

If a person can already write arbitrary code that redefines a builtin, then they can already do anything they want.
msg359457 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-01-06 19:38
> "Safe" means safe from user input to literal_eval().

Yup, apologies. I had something in mind and I realized after writing my initial comment. That is why I said afterwards:

> and the security concern is non-existent.
History
Date User Action Args
2022-04-11 14:59:24adminsetgithub: 83339
2020-12-20 19:49:09rhettingersetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2020-01-06 19:38:11pablogsalsetmessages: + msg359457
2020-01-06 19:24:07rhettingersetmessages: + msg359453
2020-01-04 15:02:35pablogsalsetmessages: + msg359292
2020-01-04 13:41:21serhiy.storchakasetstatus: closed -> open

assignee: docs@python
components: + Documentation, - Library (Lib)

nosy: + docs@python
messages: + msg359287
resolution: fixed -> (no value)
stage: resolved -> needs patch
2020-01-03 19:58:16pablogsalsetstatus: open -> closed
resolution: fixed
messages: + msg359253
2020-01-03 19:40:29pablogsalsetmessages: + msg359252
2020-01-03 19:32:37pablogsalsetmessages: + msg359251
2020-01-03 19:32:13pablogsalsetmessages: - msg359250
2020-01-03 19:31:46pablogsalsetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg359250
2020-01-03 05:21:37rhettingersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-01-03 05:21:26rhettingersetmessages: + msg359226
2019-12-29 22:35:17rhettingersetmessages: + msg359012
2019-12-29 22:08:41pablogsalsetnosy: + pablogsal
2019-12-29 21:57:06serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg359010
2019-12-29 21:48:26rhettingersetkeywords: + patch
stage: patch review
pull_requests: + pull_request17184
2019-12-29 21:48:23BTaskayasetnosy: + BTaskaya
type: enhancement
2019-12-29 21:42:10rhettingercreate