This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_fileio fails on windows MSVC Assertion
Type: behavior Stage: resolved
Components: Library (Lib), Tests Versions: Python 3.1
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, andreas.schawo, kristjan.jonsson, loewis, ocean-city, pitrou
Priority: critical Keywords:

Created on 2009-03-23 16:07 by andreas.schawo, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
Capture-QEMU.png pitrou, 2009-03-24 09:44
Messages (17)
msg84015 - (view) Author: Andreas Schawo (andreas.schawo) Date: 2009-03-23 16:07
test_fileio fails on windows with MSVC "Debug Assertion failed" message
since last week.
msg84016 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-23 16:26
Can you find out in which test case the assertion is triggered?
Also, is there more information in the error message itself?
msg84023 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-03-23 18:39
I cannot test this because I don't have recent VC, but last week change
to _fileio.c is this.

http://svn.python.org/view/python/branches/py3k/Modules/_fileio.c?r1=70187&r2=70352

On windows, lseek was used before, but _lseeki64 in portable_lseek is
used now. Maybe does following patch work?

Index: Modules/_fileio.c
===================================================================
--- Modules/_fileio.c	(revision 70537)
+++ Modules/_fileio.c	(working copy)
@@ -682,6 +682,9 @@
 			return NULL;
 	}
 
+	if (!_PyVerify_fd(fd))
+		return NULL;
+
 	Py_BEGIN_ALLOW_THREADS
 #if defined(MS_WIN64) || defined(MS_WINDOWS)
 	res = _lseeki64(fd, pos, whence);
msg84029 - (view) Author: Andreas Schawo (andreas.schawo) Date: 2009-03-23 20:29
I'm afraid your patch doesn't solve it.

I got the same popup:

Debug Assertion Failed!
Program: .....\python_d.exe
File: f:\dd\vctoools\crt_bld\self_x86\crt\src\close.c
Line: 48
Expression: (_osfile(fh) & FOPEN)

It failed while running testErrnoOnClose.

I'm using VC 2008 Express Edition.
msg84030 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-23 20:34
I wonder whether a magic assertion-disabling spell is needed around the
call to close() in _fileio.c. Kristjan, do you have any insights on this?
msg84032 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-03-23 20:59
I intentionally don't covered all the places where a fd would be needed, 
only those where the fd was delivered through python.  It is possible 
that there is some new error that is causing e.g. a double close of a 
fd.
A few months ago, when I removed the assertions experimentally, I found 
a few such bugs, related to the new IO library.
I'll look into it...  I'll have to repro since there is no traceback.
msg84034 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-23 21:05
It is not strictly speaking "an error" which is causing the double
close... It's the deliberate behaviour of a test I have added to check
that errno is correctly set on the raised IOError when calling close()
on a FileIO object whose handle has already been closed manually (with
os.close()).

In other words, since it is possible to retrieve the file descriptor of
an IO object using the fileno() method (and, moreover, it is possible to
give an existing file descriptor to the open() function so as to wrap it
inside an IO object), access to a closed file is always possible and
should be protected against (that is, it raises an IOError with the
errno attribute equal to EBADF).
msg84036 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-03-23 21:17
You are right, if it is accessible from python, then we should guard 
against it.  The case I was referring to (I'm too lazy to find the 
revision) was that a python exception wasn't raised either, so a logic 
error fell through, but the assert found it.

Anyway, that's beside the point:  The point is that I can't repro this 
using the latest version of the py3k branch, running 
lib/test/regrtest.py test_fileio
Am I in the wrong place?
msg84063 - (view) Author: Andreas Schawo (andreas.schawo) Date: 2009-03-24 08:39
I stepped back to r70187 and got no assertion.
An update to r70349 result in an assertion.

So I reverted the changes in test_fileio.py of r70349 and got no assertion.

But the test should work without assertion.

I tested releases 2.6.1, 3.0.1 and current trunk with:
import os
f = open('test.test', 'w')
os.close(f.fileno())
f.close()

2.6.1 and 3.0.1 raised an Exception with Errno 9.
With current trunk I've got the MSVC assertion.
msg84070 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-24 09:44
I can definitely reproduce the problem under a Windows XP virtual
machine using Visual C++ 2008 Express in debug mode. Here is a screenshot.

Unfortunately, I'm not a Windows programmer and I'm not able to produce
a patch for this. But the close() call in _fileio.c should certainly be
protected by whatever magic is necessary.
msg84076 - (view) Author: Andreas Schawo (andreas.schawo) Date: 2009-03-24 11:43
r69214 works with os.close(f.fileno());f.close() (Exception with Errno 9)
r69560 does not compile
r70152 test_fileio failed (different reason)
       MSVC Assertion with os.close(f.fileno());f.close()

I was not able to use _fileio.c from r69214 in cause of other dependencies.
msg84077 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-03-24 13:23
Okay, I have fixed this in revision: 70579, by adding a test in the 
close_internal() function.

However, the (somewhat pathological) case:
f = open('test.test', 'w')
os.close(f.fileno())
f.close()
Shows that there are other ways to make it crash.  By calling read, for 
example, after the os.close().  Can you add further tests and find out 
the operations that can have similar results?
msg84081 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-24 13:43
The fix is wrong, it doesn't raise an error when the fd is invalid...
msg84082 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-03-24 13:52
Sorry, fixed in 70580.
But what other cases are there?
read? write? stat?
msg84084 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-24 14:02
Basically, all FileIO methods should be protected (I'm not even sure why
you are asking! we can't let the interpreter crash rather than raise an
exception).
Someone has to write additional tests for those though, for now only
close() is tested.
msg84092 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-03-24 15:29
I´ve written tests for the other cases, and fixed them.
See revision 70582
msg110205 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-13 15:32
Closed as stated fixed in revision 70582.
History
Date User Action Args
2022-04-11 14:56:46adminsetgithub: 49794
2010-07-27 15:19:52BreamoreBoysetstatus: open -> closed
2010-07-13 15:32:06BreamoreBoysetnosy: + BreamoreBoy
messages: + msg110205

resolution: fixed
stage: resolved
2009-03-24 15:29:20kristjan.jonssonsetmessages: + msg84092
2009-03-24 14:02:49pitrousetmessages: + msg84084
2009-03-24 13:52:06kristjan.jonssonsetmessages: + msg84082
2009-03-24 13:43:31pitrousetmessages: + msg84081
2009-03-24 13:23:55kristjan.jonssonsetmessages: + msg84077
2009-03-24 11:43:26andreas.schawosetmessages: + msg84076
2009-03-24 09:44:44pitrousetfiles: + Capture-QEMU.png

messages: + msg84070
2009-03-24 08:39:47andreas.schawosetmessages: + msg84063
2009-03-23 21:17:33kristjan.jonssonsetmessages: + msg84036
2009-03-23 21:05:42pitrousetmessages: + msg84034
2009-03-23 20:59:41kristjan.jonssonsetmessages: + msg84032
2009-03-23 20:34:44pitrousetnosy: + loewis, kristjan.jonsson
messages: + msg84030
2009-03-23 20:29:38andreas.schawosetmessages: + msg84029
title: test_fileio fials on windows MSVC Assertion -> test_fileio fails on windows MSVC Assertion
2009-03-23 18:39:41ocean-citysetnosy: + ocean-city
messages: + msg84023
2009-03-23 16:26:54pitrousetpriority: critical
type: behavior
messages: + msg84016

components: + Library (Lib)
2009-03-23 16:07:31andreas.schawocreate