Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[subinterpreters] Unicode interned strings must not be shared between interpreters #90441

Closed
vstinner opened this issue Jan 6, 2022 · 3 comments
Labels

Comments

@vstinner
Copy link
Member

vstinner commented Jan 6, 2022

BPO 46283
Nosy @vstinner, @encukou, @ericsnowcurrently

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-01-06.15:21:22.175>
labels = ['expert-subinterpreters', '3.11']
title = '[subinterpreters] Unicode interned strings must not be shared between interpreters'
updated_at = <Date 2022-01-10.10:50:20.435>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2022-01-10.10:50:20.435>
actor = 'petr.viktorin'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Subinterpreters']
creation = <Date 2022-01-06.15:21:22.175>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 46283
keywords = []
message_count = 2.0
messages = ['409860', '410197']
nosy_count = 3.0
nosy_names = ['vstinner', 'petr.viktorin', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue46283'
versions = ['Python 3.11']

@vstinner
Copy link
Member Author

vstinner commented Jan 6, 2022

My commit ea25180 made Unicode interned strings per interpreter to avoid accessing the same Python object from two different interpreters, to avoid race conditions on its reference count.

Problem: the change introduced the bpo-46006 regression in _PyUnicode_EqualToASCIIId() and type update_slot() functions which make the assumption that if two strings are interned and their pointers are not equal: these strings are not equal.

I proposed PR 30123 to fix these two functions, but then questions have been asked about the overall goal, running multiple Python interpreters in parallel in the same process (bpo-40512):
https://bugs.python.org/issue46006#msg408002

bpo-46006 was blocking the Python 3.11.0a4 release, Python 3.10 was affected and there were more and more affected projects:

So I decided to just revert the change on interned strings:

  • main branch: commit 35d6540
  • 3.10 branch: commit 72c260c (with changes to keep the ABI backward compatibility)

These interned strings are preventing again to run multiple interpreters in parallel, two subinterpreters must not access the same Python object: see bpo-40533.

I suggest to write a PEP to explain the rationale, and reapply my commit ea25180 with PR 30123.

@vstinner vstinner added 3.11 only security fixes topic-subinterpreters labels Jan 6, 2022
@encukou
Copy link
Member

encukou commented Jan 10, 2022

Another proposed alternative is to make these immortal, and share them.

There were some discussions about this, but no PEP to capture the pros/cons of the alternatives and the decision.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@vstinner
Copy link
Member Author

vstinner commented Nov 3, 2022

Since most core developers seem to prefer https://peps.python.org/pep-0683/ (even if it's still a draft), I close this issue.

@vstinner vstinner closed this as completed Nov 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

No branches or pull requests

2 participants