Title: Fix constness of members of tok_state struct.
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.9
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, petdance, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2020-02-22 00:25 by petdance, last changed 2020-02-28 02:45 by benjamin.peterson. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 18600 merged petdance, 2020-02-22 00:28
Messages (2)
msg362441 - (view) Author: Andy Lester (petdance) * Date: 2020-02-22 00:25
The function PyTokenizer_FromUTF8 from Parser/tokenizer.c had a comment:

    /* XXX: constify members. */

This patch addresses that.

In the tok_state struct:
    * end and start were non-const but could be made const
    * str and input were const but should have been non-const

Changes to support this include:
    * decode_str() now returns a char * since it is allocated.
    * PyTokenizer_FromString() and PyTokenizer_FromUTF8() each creates a
        new char * for an allocate string instead of reusing the input
        const char *.
    * PyTokenizer_Get() and tok_get() now take const char ** arguments.
    * Various local vars are const or non-const accordingly.

I was able to remove five casts that cast away constness.
msg362859 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2020-02-28 02:45
New changeset 384f3c536dd15ba33ea7e8afb4087ae359d4c12e by Andy Lester in branch 'master':
closes bpo-39721: Fix constness of members of tok_state struct. (GH-18600)
Date User Action Args
2020-02-28 02:45:16benjamin.petersonsetstatus: open -> closed

nosy: + benjamin.peterson
messages: + msg362859

resolution: fixed
stage: patch review -> resolved
2020-02-22 07:30:36SilentGhostsetnosy: + vstinner, serhiy.storchaka

type: behavior
versions: + Python 3.9
2020-02-22 00:28:35petdancesetkeywords: + patch
stage: patch review
pull_requests: + pull_request17967
2020-02-22 00:25:34petdancecreate