This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients gregory.p.smith, melwitt, vinay.sajip, vstinner
Date 2020-04-07.23:12:37
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1586301157.31.0.715065478746.issue40091@roundup.psfhosted.org>
In-reply-to
Content
> because I encountered a problem where a standard library lock was held by a parent process at the time that child processes were forked, so the child processes got stuck behind the inherited held locks.

Which lock from which module? You wrote details at:
https://github.com/python/cpython/pull/19195#issuecomment-609583084

According to your comment #28 at https://bugs.launchpad.net/nova/+bug/1844929 the lock involved in the issue comes from _TransactionFactory of oslo.db:
https://github.com/openstack/oslo.db/blob/b903d4e1ee07ef2ec454daa5b8418b3039e02774/oslo_db/sqlalchemy/enginefacade.py#L189

So it's a bug in oslo.db, not in Python. Python doesn't provide any machinery to automatically reinitialize all locks created in Python at fork in the child process. os.register_at_fork() must be used explicitly.


> But, if I'm understanding correctly, this issue is fixing something in python logging specifically and not all standard library locks in general.

This issue is specific to logging.


> My question is, will there be a way to reinit standard library locks in general using _at_fork_reinit()? That is, should we expect a future fix in python to do this or is the recommendation to continue to ensure the application reinits locks during process start if we know the process could be a child?

Each module has to setup an os.register_at_fork() callback to reinitialize its locks. It's done by threading and logging modules for example. The multiprocessing has its own util.register_after_fork() machinery (see bpo-40221)..
History
Date User Action Args
2020-04-07 23:12:37vstinnersetrecipients: + vstinner, gregory.p.smith, vinay.sajip, melwitt
2020-04-07 23:12:37vstinnersetmessageid: <1586301157.31.0.715065478746.issue40091@roundup.psfhosted.org>
2020-04-07 23:12:37vstinnerlinkissue40091 messages
2020-04-07 23:12:37vstinnercreate