New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
logging._removeHandlerRef is not threadsafe during interpreter shutdown #65348
Comments
If another thread is active during interpreter shutdown, it can hold the last reference to a handler; when it drops that reference, the weakref callback -- _removeHandlerRef -- will be executed in this other thread. So while this callback is running, the main thread is replacing module globals with None. This creates a data race for the globals in logging -- for example, _releaseLock can be replaced with None after the "_releaseLock is not None" check, but before it is used. In principle I suspect this could cause a deadlock, in practice all I've seen are exception messages mentioning how None is not callable. I have attached a patch that I think resolves this issue. The patch is written against 2.7, although I expect this issue affects all versions of Python prior to 3.4 BTW, the copyright for this patch belongs to my employer, Google; please let me know if there are any contributor agreements or such that my employer needs to look at. |
Yes, there is a contributor agreement form which needs to be completed and signed: |
Are you sure? There should have been many previous contributions by Google, so the relevant copyright agreements _should_ have already been signed. I asked internally and was told that a corporate version of this agreement had been signed a long time ago. |
Yes, I remember previous discussions of the corporate agreement from Google, so I'm sure it exists. |
New changeset b6deab7204e6 by Vinay Sajip in branch '2.7': New changeset b5c91b61991a by Vinay Sajip in branch '3.4': New changeset 76689a706900 by Vinay Sajip in branch 'default': |
Please don't take my word for it, but my understanding is that this issue doesn't apply to 3.4+ since module globals are no longer set to None during interpreter shutdown. (So all the checks against None could even be deleted.) |
The release notes say about PEP-442 that "As part of this change, module globals are no longer forcibly set to None during interpreter shutdown in most cases". I haven't looked at the PEP in detail, so I'm not sure specifically what "most cases" means - so I played it safe. |
We're still seeing this (in 2.7.12, but the code at 2.7 head hasn't changed). In this case the RLock being used within _acquireLock() still exists but is an incomplete object by the time we use it. |
Here is a minimal example: main.py: It does not happen in Python 3 though (3.5.3 at least). |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: