Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ast.literal_eval() doesn't support empty sets #83339

Closed
rhettinger opened this issue Dec 29, 2019 · 11 comments
Closed

ast.literal_eval() doesn't support empty sets #83339

rhettinger opened this issue Dec 29, 2019 · 11 comments
Labels
3.9 only security fixes docs Documentation in the Doc dir type-feature A feature request or enhancement

Comments

@rhettinger
Copy link
Contributor

BPO 39158
Nosy @rhettinger, @serhiy-storchaka, @pablogsal, @isidentical
PRs
  • bpo-39158: ast.literal_eval() doesn't support empty sets #17742
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-12-20.19:49:09.236>
    created_at = <Date 2019-12-29.21:42:10.401>
    labels = ['type-feature', '3.9', 'docs']
    title = "ast.literal_eval() doesn't support empty sets"
    updated_at = <Date 2020-12-20.19:49:09.235>
    user = 'https://github.com/rhettinger'

    bugs.python.org fields:

    activity = <Date 2020-12-20.19:49:09.235>
    actor = 'rhettinger'
    assignee = 'docs@python'
    closed = True
    closed_date = <Date 2020-12-20.19:49:09.236>
    closer = 'rhettinger'
    components = ['Documentation']
    creation = <Date 2019-12-29.21:42:10.401>
    creator = 'rhettinger'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 39158
    keywords = ['patch']
    message_count = 11.0
    messages = ['359007', '359010', '359012', '359226', '359251', '359252', '359253', '359287', '359292', '359453', '359457']
    nosy_count = 5.0
    nosy_names = ['rhettinger', 'docs@python', 'serhiy.storchaka', 'pablogsal', 'BTaskaya']
    pr_nums = ['17742']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue39158'
    versions = ['Python 3.9']

    @rhettinger
    Copy link
    Contributor Author

    We already support sets but not empty sets. After the PR, this now works:

        >>> from ast import literal_eval
        >>> literal_eval('set()')
        set()

    If we wanted, it would be a simple matter to extend it frozensets:

        >>> literal_eval('frozenset({10, 20, 30})')
        frozenset({10, 20, 30})

    @rhettinger rhettinger added stdlib Python modules in the Lib dir 3.9 only security fixes labels Dec 29, 2019
    @isidentical isidentical added type-feature A feature request or enhancement labels Dec 29, 2019
    @serhiy-storchaka
    Copy link
    Member

    set() is neither literal nor container display.

    @rhettinger
    Copy link
    Contributor Author

    set() is neither literal nor container display.

    Yes, that is obvious. However, we do support sets and set() is how make an empty set.

    It is weird to support sets but not empty sets, especially when it is so easy to do so safely.

    @rhettinger
    Copy link
    Contributor Author

    New changeset 4fcf5c1 by Raymond Hettinger in branch 'master':
    bpo-39158: ast.literal_eval() doesn't support empty sets (GH-17742)
    4fcf5c1

    @pablogsal pablogsal reopened this Jan 3, 2020
    @pablogsal pablogsal reopened this Jan 3, 2020
    @pablogsal
    Copy link
    Member

    The function literal_eval is not safe anymore as the constructor can be intercepted:

    >>> import builtins
    >>> def evil_code(*args):
    ...     print("Something evil")
    ...
    >>> builtins.set = evil_code
    >>> ast.literal_eval("set()")
    Something evil

    I think we should either use {0}.__class__.

    Also, the documentation now is wrong as the function does more than evaluate literals or container displays.

    @pablogsal
    Copy link
    Member

    The function literal_eval is not safe anymore as the constructor can be intercepted:

    Well, actually it can also be done with any other builtin via the same trick, so this is no different but is only slightly more obvious that it can be done.

    I still find a bit weird that the documentation says "literal or container display" but 'set' is neither.

    @pablogsal
    Copy link
    Member

    I am re-closing the issue as I don't think is worth complicating the docs for this edge case and the security concern is non existent.

    Apologies for the noise. If someone feels strongly about the documentation, they can reopen the issue.

    @serhiy-storchaka
    Copy link
    Member

    The documentation for ast.literal_eval():

    Safely evaluate an expression node or a string containing a Python literal or
    container display. The string or node provided may only consist of the
    following Python literal structures: strings, bytes, numbers, tuples, lists,
    dicts, sets, booleans, and None.

    https://docs.python.org/3/library/ast.html#ast.literal_eval

    If we are going to add support of a function call set(), we should change the documentation.

    And if add support of non-literals, where should we stop? Should we support also frozenset() and bytearray()? inf and nan? infj and nanj? complex()? Ellipsis? __debug__?

    @serhiy-storchaka serhiy-storchaka added docs Documentation in the Doc dir and removed stdlib Python modules in the Lib dir labels Jan 4, 2020
    @serhiy-storchaka serhiy-storchaka added docs Documentation in the Doc dir and removed stdlib Python modules in the Lib dir labels Jan 4, 2020
    @pablogsal
    Copy link
    Member

    And if add support of non-literals, where should we stop? Should we support also frozenset() and bytearray()? inf and nan? infj and nanj? complex()? Ellipsis? __debug__?

    Then the name of the function would be a bit misleading (for frozenset() and bytearray()), no? I think the set() change is somehow inconsistent already with the function name.

    @rhettinger
    Copy link
    Contributor Author

    The function literal_eval is not safe anymore as the
    constructor can be intercepted

    "Safe" means safe from user input to literal_eval().

    If a person can already write arbitrary code that redefines a builtin, then they can already do anything they want.

    @pablogsal
    Copy link
    Member

    "Safe" means safe from user input to literal_eval().

    Yup, apologies. I had something in mind and I realized after writing my initial comment. That is why I said afterwards:

    and the security concern is non-existent.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes docs Documentation in the Doc dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants