classification
Title: TemporaryDirectory is cleaned up twice
Type: behavior Stage: resolved
Components: Versions: Python 3.4
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Ilya.Kulakov, rbcollins
Priority: normal Keywords:

Created on 2015-07-23 22:51 by Ilya.Kulakov, last changed 2017-01-10 03:32 by eryksun. This issue is now closed.

Messages (2)
msg247233 - (view) Author: Ilya Kulakov (Ilya.Kulakov) * Date: 2015-07-23 22:51
I'm seeing the issue using python 3.4.3 on Windows 8

In the __init__.py method of my package I define temporary directory at the module level like this:

_TempDir = tempfile.TemporaryDirectory(prefix='...'))
tempfile.tempdir = _TempDir.name


I expect it to be deleted on exit.

However, _sometimes_, I'm seeing the following exception on exit:

Traceback (most recent call last):
  File ":/weakref.pyo", line 582, in _exitfunc
  File ":/weakref.pyo", line 506, in __call__
  File ":/tempfile.pyo", line 674, in _cleanup
  File ":/shutil.pyo", line 478, in rmtree
  File ":/shutil.pyo", line 360, in _rmtree_unsafe
  File ":/shutil.pyo", line 358, in _rmtree_unsafe
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\
Users\\BC79~1\\AppData\\Local\\Temp\\..._s9wqmyo_'
msg247287 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-07-24 15:42
I think you may need to instrument TemporaryDirectory._cleanup to be sure, but it sounds like its being run twice.

now, you're not using it like a context manager (at least as far as your code shows), so it must be happening from the weakref.

https://docs.python.org/3/library/weakref.html#weakref.finalize is the relevant docs for that.

The code looks ok as long as finalize triggers once and only once. Perhaps it should call the finalize() rather than manually calling _cleanup, in cleanup, but I don't see that that should make much difference. I would have thought it a deliberate attempt to avoid some bit of code (e.g. the resource warning), but since its a shared helper, thats not it.

And finalize._exitfunc looks entirely sane to me.

So - I suggest adding a call to print_stack in TemporaryDirectory._cleanup, to see the entire stack, and then hopefully we'll see two such printouts when this error happens, and be able to pinpoint how it's being called twice.
History
Date User Action Args
2017-01-10 03:32:54eryksunsetresolution: out of date
stage: resolved
2017-01-10 01:38:07Ilya.Kulakovsetstatus: open -> closed
2015-07-24 15:42:42rbcollinssetnosy: + rbcollins
messages: + msg247287
2015-07-23 22:51:03Ilya.Kulakovcreate