classification
Title: Windows: with-statement doesn't release file handle after exception on "No space left on device" error
Type: behavior Stage: resolved
Components: IO, Windows Versions: Python 3.2
process
Status: closed Resolution: out of date
Dependencies: Superseder: file descriptor not being closed with context manager on IOError when device is full
View: 16597
Assigned To: Nosy List: Lauri Kajan, eryksun, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords:

Created on 2015-09-21 13:17 by Lauri Kajan, last changed 2015-09-21 18:02 by Lauri Kajan. This issue is now closed.

Messages (5)
msg251225 - (view) Author: Lauri Kajan (Lauri Kajan) Date: 2015-09-21 13:17
Hi all,

I found out that handle to a file opened with with-statement is not released in all cases.

I'm trying to delete a file when space on a disk is filled (0 bit left) by writing this file. I catch the IOError 28 to figure this out.
I write the file within the with-statement that is wrapped with try except. In the except block I try to delete the file but the handle to the file is not released.

In the following code fill-file can't be deleted because the file is still open.
I have described the problem also in stackoverflow [1].

# I recommend filling the disk almost full with some better tool.
import os
import errno

fill = 'J:/fill.txt'
try:
    with open(fill, 'wb') as f:
        while True:
            n = f.write(b"\0")
except IOError as e:
    if e.errno == errno.ENOSPC:
        os.remove(fill)

This gives the following traceback:

Traceback (most recent call last):
  File "nospacelef.py", line 8, in <module>
    n = f.write(b"\0")
IOError: [Errno 28] No space left on device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "nospacelef.py", line 8, in <module>
    n = f.write(b"\0")
IOError: [Errno 28] No space left on device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "nospacelef.py", line 11, in <module>
    os.remove(fill)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'J:/fill.txt'



[1] http://stackoverflow.com/questions/32690018/cant-delete-a-file-after-no-space-left-on-device
msg251229 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-09-21 14:57
See issue 16597. This wasn't fixed for 3.2, so it won't close the file if flush() fails. You'll either have to work around this or upgrade to 3.3+.
msg251232 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-09-21 15:22
@Lauri: What is your Python version? Can you retry your test with Python 3.5?
msg251234 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-09-21 15:54
Lauri changed the version from 3.4 to 3.2, and I was able to reproduce the problem in 3.2.5:

    C:\Temp>py -3.2 --version
    Python 3.2.5
    C:\Temp>py -3.2 nospace.py
    removing fill.txt
    Traceback (most recent call last):
      File "nospace.py", line 8, in <module>
        n = f.write(b"\0")
    IOError: [Errno 28] No space left on device

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "nospace.py", line 8, in <module>
        n = f.write(b"\0")
    IOError: [Errno 28] No space left on device

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "nospace.py", line 12, in <module>
        os.remove(fill)
    WindowsError: [Error 32] The process cannot access the file because it is 
    being used by another process: 'T:\\fill.txt'

It's a sharing violation, since the file handle isn't closed and wasn't opened with FILE_SHARE_DELETE.

There's no problem in 3.4 or 3.5 since it properly closes the file handle even if flush() fails:

    C:\Temp>py -3.4 --version
    Python 3.4.3
    C:\Temp>py -3.4 nospace.py
    removing fill.txt

    C:\Temp>py -3.5 --version
    Python 3.5.0
    C:\Temp>py -3.5 nospace.py
    removing fill.txt
msg251245 - (view) Author: Lauri Kajan (Lauri Kajan) Date: 2015-09-21 18:02
Thanks for these.
I'll see what I could do to upgrade our python version.

Sorry for not finding the old issue. Didn't find the right search terms.
History
Date User Action Args
2015-09-21 18:02:55Lauri Kajansetmessages: + msg251245
2015-09-21 15:54:38eryksunsetmessages: + msg251234
2015-09-21 15:22:40vstinnersetnosy: + vstinner
messages: + msg251232
2015-09-21 15:03:28zach.waresetsuperseder: file descriptor not being closed with context manager on IOError when device is full
2015-09-21 14:57:46eryksunsetstatus: open -> closed

nosy: + eryksun
messages: + msg251229

resolution: out of date
stage: resolved
2015-09-21 13:44:56Lauri Kajansetversions: + Python 3.2, - Python 3.4
2015-09-21 13:19:43vstinnersetnosy: + paul.moore, tim.golden, zach.ware, steve.dower
components: + Windows
2015-09-21 13:19:36vstinnersettitle: with-statement doesn't release file handle after exception -> Windows: with-statement doesn't release file handle after exception on "No space left on device" error
2015-09-21 13:17:04Lauri Kajancreate