Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C API] Deprecate legacy API for configure Python initialization #88279

Closed
vstinner opened this issue May 12, 2021 · 11 comments
Closed

[C API] Deprecate legacy API for configure Python initialization #88279

vstinner opened this issue May 12, 2021 · 11 comments
Labels
3.11 only security fixes docs Documentation in the Doc dir topic-C-API

Comments

@vstinner
Copy link
Member

BPO 44113
Nosy @vstinner, @corona10, @joukewitteveen
PRs
  • bpo-44113: Deprecate old functions to config Python init #26060
  • bpo-43956: Doc/c-api: Fix default program name #24876
  • bpo-44113: Update __xxtestfuzz not to use Py_SetProgramName #26083
  • bpo-44113: Fix compiler warning in PySys_AddWarnOption() #26084
  • bpo-44113: Update fromzenmain not to use Py_SetProgramName #26085
  • bpo-44113: Move the What's New entry to Deprecate section #28974
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-05-14.12:21:54.201>
    created_at = <Date 2021-05-12.10:20:54.214>
    labels = ['expert-C-API', '3.11', 'docs']
    title = '[C API] Deprecate legacy API for configure Python initialization'
    updated_at = <Date 2021-10-15.11:43:51.791>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-10-15.11:43:51.791>
    actor = 'vstinner'
    assignee = 'docs@python'
    closed = True
    closed_date = <Date 2021-05-14.12:21:54.201>
    closer = 'vstinner'
    components = ['Documentation', 'C API']
    creation = <Date 2021-05-12.10:20:54.214>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44113
    keywords = ['patch']
    message_count = 8.0
    messages = ['393499', '393550', '393551', '393553', '393641', '393647', '393648', '404005']
    nosy_count = 4.0
    nosy_names = ['vstinner', 'docs@python', 'corona10', 'joukewitteveen']
    pr_nums = ['26060', '24876', '26083', '26084', '26085', '28974']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue44113'
    versions = ['Python 3.11']

    @vstinner
    Copy link
    Member Author

    The https://docs.python.org/dev/c-api/init.html documentation lists many functions which is the legacy way to configure the Python initialization. These functions are kept for backward compatibility but have flaws and are less reliable than the new PyConfig API (PEP-587) documented at https://docs.python.org/dev/c-api/init_config.html

    I propose to deprecate the legacy functions to configure the Python initialization. Examples:

    • Py_SetPath()
    • Py_SetProgramName()
    • Py_SetPythonHome()
    • Py_SetStandardStreamEncoding()
    • PySys_AddWarnOption()
    • PySys_AddWarnOptionUnicode()
    • PySys_AddXOption()

    I don't propose to schedule the removal of these functions, only mark them as deprecated in the *documentation*.

    Related issue: bpo-43956 "C-API: Incorrect default value for Py_SetProgramName" and PR 24876.

    @vstinner vstinner added the 3.11 only security fixes label May 12, 2021
    @vstinner vstinner added docs Documentation in the Doc dir topic-C-API 3.11 only security fixes labels May 12, 2021
    @vstinner vstinner added docs Documentation in the Doc dir topic-C-API labels May 12, 2021
    @vstinner
    Copy link
    Member Author

    PyConfig API was added to Python 3.8, so it's now widely available. Python 3.8 always switched to security fixes only phase. So IMO it's ok to deprecate the old API now.

    @vstinner
    Copy link
    Member Author

    """
    Check warning on line 82 in Python/frozenmain.c
    GitHub Actions / Ubuntu
    ‘Py_SetProgramName’ is deprecated [-Wdeprecated-declarations]
    """

    Well, Py_FrozenMain() should be rewritten with PyConfig, I want to do that for a long time, but I don't know how to test it. It would be nice to have a very basic test for Py_FrozenMain().

    See also:
    https://mail.python.org/archives/list/capi-sig@python.org/thread/5QLI3NUP3OSGLCCIBAQOTX4GEJQBWJ6F/

    @vstinner
    Copy link
    Member Author

    commit 6cd0446
    Author: Victor Stinner <vstinner@python.org>
    Date: Wed May 12 23:59:25 2021 +0200

    bpo-44113: Deprecate old functions to config Python init (GH-26060)
    
    Deprecate the following functions to configure the Python
    initialization:
    
    * PySys_AddWarnOption()
    * PySys_AddWarnOptionUnicode()
    * PySys_AddXOption()
    * PySys_HasWarnOptions()
    * Py_SetPath()
    * Py_SetProgramName()
    * Py_SetPythonHome()
    * Py_SetStandardStreamEncoding()
    * _Py_SetProgramFullPath()
    
    Use the new PyConfig API of the Python Initialization Configuration
    instead (PEP-587).
    

    commit a0ccc40
    Author: Dong-hee Na <donghee.na@python.org>
    Date: Thu May 13 08:22:18 2021 +0900

    bpo-44113: Update __xxtestfuzz not to use Py_SetProgramName (GH-26083)
    

    commit ec7c09b
    Author: Victor Stinner <vstinner@python.org>
    Date: Thu May 13 02:27:56 2021 +0200

    bpo-44113: Fix compiler warning in PySys_AddWarnOption() (GH-26084)
    
    Ignore Py_DEPRECATED() warning on purpose.
    

    @vstinner
    Copy link
    Member Author

    commit 7565586
    Author: Dong-hee Na <donghee.na@python.org>
    Date: Thu May 13 10:19:46 2021 +0900

    bpo-44113: Update fromzenmain not to use Py_SetProgramName (GH-26085)
    

    @vstinner
    Copy link
    Member Author

    Well, Py_FrozenMain() should be rewritten with PyConfig, I want to do that for a long time, but I don't know how to test it. It would be nice to have a very basic test for Py_FrozenMain().

    I created bpo-44131: [C API] Add tests on Py_FrozenMain().

    @vstinner
    Copy link
    Member Author

    Thanks for the help Dong-hee ;-) I close the issue, functions are now deprecated.

    @vstinner
    Copy link
    Member Author

    New changeset a7f8dfd by Victor Stinner in branch 'main':
    bpo-44113: Move the What's New entry to Deprecate section (GH-28974)
    a7f8dfd

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @vstinner vstinner reopened this May 6, 2022
    vstinner added a commit that referenced this issue May 6, 2022
    Deprecate the following C functions:
    
    * PySys_SetArgv()
    * PySys_SetArgvEx()
    * PySys_SetPath()
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue May 7, 2022
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 8, 2022
    …gvEx (pythonGH-92428)
    
    (cherry picked from commit bd030b6)
    
    Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
    miss-islington added a commit that referenced this issue May 8, 2022
    …H-92428)
    
    (cherry picked from commit bd030b6)
    
    Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
    saagarjha pushed a commit to ahjragaas/binutils-gdb that referenced this issue Jul 20, 2022
    Python 3.11 deprecates PySys_SetPath and Py_SetProgramName.  The
    PyConfig API replaces these and other functions.  This commit uses the
    PyConfig API to provide equivalent functionality while also preserving
    support for older versions of Python, i.e. those before Python 3.8.
    
    A beta version of Python 3.11 is available in Fedora Rawhide.  Both
    Fedora 35 and Fedora 36 use Python 3.10, while Fedora 34 still used
    Python 3.9.  I've tested these changes on Fedora 34, Fedora 36, and
    rawhide, though complete testing was not possible on rawhide due to
    a kernel bug.  That being the case, I decided to enable the newer
    PyConfig API by testing PY_VERSION_HEX against 0x030a0000.  This
    corresponds to Python 3.10.
    
    We could try to use the PyConfig API for Python versions as early as 3.8,
    but I'm reluctant to do this as there may have been PyConfig related
    bugs in earlier versions which have since been fixed.  Recent linux
    distributions should have support for Python 3.10.  This should be
    more than adequate for testing the new Python initialization code in
    GDB.
    
    Information about the PyConfig API as well as the motivation behind
    deprecating the old interface can be found at these links:
    
    python/cpython#88279
    https://peps.python.org/pep-0587/
    https://docs.python.org/3.11/c-api/init_config.html
    
    The v2 commit also addresses several problems that Simon found in
    the v1 version.
    
    In v1, I had used Py_DontWriteBytecodeFlag in the new initialization
    code, but Simon pointed out that this global configuration variable
    will be deprecated in Python 3.12.  This version of the patch no longer
    uses Py_DontWriteBytecodeFlag in the new initialization code.
    Additionally, both Py_DontWriteBytecodeFlag and Py_IgnoreEnvironmentFlag
    will no longer be used when building GDB against Python 3.10 or higher.
    While it's true that both of these global configuration variables are
    deprecated in Python 3.12, it makes sense to disable their use for
    gdb builds against 3.10 and higher since those are the versions for
    which the PyConfig API is now being used by GDB.  (The PyConfig API
    includes different mechanisms for making the same settings afforded
    by use of the soon-to-be deprecated global configuration variables.)
    
    Simon also noted that PyConfig_Clear() would not have be called for
    one of the failure paths.  I've fixed that problem and also made the
    rest of the "bail out" code more direct.  In particular,
    PyConfig_Clear() will always be called, both for success and failure.
    
    The v3 patch addresses some rebase conflicts related to module
    initialization .  Commit 3acd9a6 ("Make 'import gdb.events' work")
    uses PyImport_ExtendInittab instead of PyImport_AppendInittab.  That
    commit also initializes a struct for each module to import.  Both the
    initialization and the call to were moved ahead of the ifdefs to avoid
    having to replicate (at least some of) the code three times in various
    portions of the ifdefs.
    
    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28668
    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=29287
    @vstinner vstinner closed this as completed Nov 3, 2022
    @milianw
    Copy link

    milianw commented Jan 10, 2024

    Hey there, I'm unsure if this is the right place to ask - if not, please guide me elsewhere.

    How are projects using the limited python API supposed to port away from the deprecated API? The "new" API in cpython/initconfig.h is guarded behind a #ifndef Py_LIMITED_API and thus not accessible...

    Can you please make this API available through the limited API, or revert the decision to deprecate this API?

    Thanks

    @encukou
    Copy link
    Member

    encukou commented Jan 16, 2024

    The proper place is here: #105145
    Apparently, the new API is available widely enough.

    @vstinner
    Copy link
    Member Author

    How are projects using the limited python API supposed to port away from the deprecated API?

    I suggest to continue in the dedicated issue: #107954 "No limited C API to customize Python initialization (PyConfig, PEP 587)"

    I worked on two projects for the limited C API:

    So far, I had to put this work aside since I had too many things on my plate.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes docs Documentation in the Doc dir topic-C-API
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants