classification
Title: Tests failures using -u largefile when the disk is full
Type: Stage: resolved
Components: Tests Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: jkloth, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017-09-27 12:29 by vstinner, last changed 2018-05-29 21:59 by vstinner. This issue is now closed.

Messages (7)
msg303130 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-27 12:29
I ran the Python test suite on Windows on a freshly installed Python 3.7a1 and I got errors: bpo-31606.

I created a separated issue for tests failing because my disk has low free space.


On my Windows VM, I have a single 40 GB disk with 1.36 GB free.

C:\Users\haypo\AppData\Local\Programs\Python\Python37>python -m test -v -u all test_largefile

======================================================================
ERROR: test_truncate (test.test_largefile.PyLargeFileTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\test\test_largefile.py", line 37, in setUp
    f.flush()
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\_pyio.py", line 1226, in flush
    self._flush_unlocked()
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\_pyio.py", line 1233, in _flush_unlocked
    n = self.raw.write(self._write_buf)
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\_pyio.py", line 1630, in write
    return os.write(self._fd, b)
OSError: [Errno 28] No space left on device

(other tests fail with similar message)



test_io:

======================================================================
ERROR: test_large_file_ops (test.test_io.CIOTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\test\test_io.p
y", line 574, in test_large_file_ops
    self.large_file_ops(f)
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\test\test_io.p
y", line 355, in large_file_ops
    self.assertEqual(f.write(b"xxx"), 3)
OSError: [Errno 28] No space left on device

======================================================================
ERROR: test_large_file_ops (test.test_io.PyIOTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\test\test_io.py", line 574, in test_large_file_ops
    self.large_file_ops(f)
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\test\test_io.py", line 355, in large_file_ops
    self.assertEqual(f.write(b"xxx"), 3)
  File "C:\Users\haypo\AppData\Local\Programs\Python\Python37\lib\_pyio.py", line 1630, in write
    return os.write(self._fd, b)
OSError: [Errno 28] No space left on device
msg303137 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-27 13:12
This is expected behavior, isn't?
msg303140 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-27 13:28
> This is expected behavior, isn't?

I would prefer that the test skips itself when the disk is full: on "OSError: [Errno 28] No space left on device".

Or maybe largefile resource should be excluded from "all"... at least on Windows, since NTFS doesn't see to support files full of zeros without accounting zero bytes (seek + write 1 byte: NTFS counts the seek in the disk space).
msg303150 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-27 14:21
Definitely the test shouldn't be quietly skipped! If you enable the "largefile" resource, but the test can't write large file, this is either a bug in the tested code, or an error in the test, or misconfiguration.

We already have the "extralargefile" resource which is excluded from "all". "largefile" is for moderately large files, around 2 GiB.
msg303158 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-27 15:55
> "largefile" is for moderately large files, around 2 GiB.

largefile creates files larger than 2 GB. Extract of test_largefile:

# size of file to create (>2GB; 2GB == 2147483648 bytes)
size = 2500000000

My VM has less than 2 GB free: only 1.36 GB free.
msg303160 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-27 16:38
largefile - It is okay to run some test that may create huge
                files.  These tests can take a long time and may
                consume >2GB of disk space temporarily.

I understand this as 2GB is a minimum free disk space for tests without "largefile". Tests with "largefile" need more free disk space.


Accordingly the bigmemtest decorator is used for tests that consume >1GB of RAM.
msg318112 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-05-29 21:59
According to Serhiy's comment, largefile shouldn't be used with less than 2 GB of free disk space. It was my fault, it's not a bug ;-)
History
Date User Action Args
2018-05-29 21:59:05vstinnersetstatus: open -> closed
resolution: not a bug
messages: + msg318112

stage: resolved
2017-09-27 16:38:11serhiy.storchakasetmessages: + msg303160
2017-09-27 15:55:46vstinnersetmessages: + msg303158
2017-09-27 14:46:43jklothsetnosy: + jkloth
2017-09-27 14:21:29serhiy.storchakasetmessages: + msg303150
2017-09-27 13:28:38vstinnersetmessages: + msg303140
2017-09-27 13:12:03serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg303137
2017-09-27 12:29:24vstinnercreate