This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author milestonejxd
Recipients milestonejxd
Date 2022-01-15.18:04:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
Repo is the standard tool that Google uses to build Android, Chrome OS, Chromium, etc, written in Python. 

Many Repo users have encountered resource leak warnings with Python 3.9.

I did some work and found that the problem is not caused by the code of Repo, but a bug of the Python library, multiprocess.

To make it simple, the Python script below leaks named resource even when exiting normally. (And be unlinked by resource_tracker with a warning. )

import multiprocessing as mp

global_resource = mp.Semaphore()

def submain(): pass

if __name__ == '__main__':
    p = mp.Process(target=submain)

Tested on macOS with Python 3.9.7
> python
resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown.

This bug will 100% reproduce when then main module uses named resources as global variables and uses `spawn` context, which is the case of Repo on macOS.

This is caused by multiprocess::BaseProcess::_bootstrap.

When a new process is started with multiprocessing.Process.start() in `spawn` context.
1. The main module is reloaded in the subprocess (for pickle) in multiprocessing::spawn::_main.
2. Named resources (such as the semaphore above) in the main module resister their _cleanup into multiprocessing::util::_finalizer_registry, which unlink themselves.
3. multiprocess::BaseProcess::_bootstrap then clears _finalizer_registry.

When a subprocess is spawned, it is no need to clear util::_finalizer_registry (and no need to call util::_run_after_forkers). Disable clearing _finalizer_registry (and disable call to _run_after_forkers) should fix this bug without breaking anything else.

And I uploaded a MR.
Date User Action Args
2022-01-15 18:04:57milestonejxdsetrecipients: + milestonejxd
2022-01-15 18:04:57milestonejxdsetmessageid: <>
2022-01-15 18:04:57milestonejxdlinkissue46391 messages
2022-01-15 18:04:57milestonejxdcreate