classification
Title: Crash on shutdown after os.fdopen(2) in debug builds
Type: Stage: resolved
Components: Windows Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: kristjan.jonsson Nosy List: Ramchandra Apte, amaury.forgeotdarc, anselm.kruis, iritkatriel, kristjan.jonsson, pitrou
Priority: normal Keywords:

Created on 2009-04-16 12:39 by amaury.forgeotdarc, last changed 2020-11-30 14:23 by iritkatriel. This issue is now closed.

Messages (10)
msg86026 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-04-16 12:39
With a 2.6 or 2.7 debug build:
    python_d -c "import os; os.fdopen(2)"

Displays the famous Debug Assertion Failure:
    ...
    Expression: (_osfile(fh) & FOPEN)
    ...

The error occurs when closing pyhon, in call_ll_exitfuncs() there is a
call to fflush(stderr)
msg86027 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-04-16 12:51
Well, the short answer is don´t do that!

Do os.fdopen(os.fdup(2))
You are stealing the fd 2 away from stderr, then closing it, and stderr 
now holds a defunct file descriptor.
msg86035 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-04-16 16:18
Sure. But we tried hard to remove all these assertion failures, so that 
python never shows a popup dialog.
msg86065 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-04-17 10:18
Yes.  We have tried to accomodate the most reckless use of file 
descriptors.
I'm not sure how to proceed here though, since the builtin streams are 
beyond our control.

I think you would have the same effect if you called os.close(2).

Perhaps the only really good solution is to allow for disabling the fd 
checks after all, but to make it optional, somehow.  Ideally something 
that python.exe sets up, it being the prime motivator....

Really, this fd checking in mscrt is becoming really annoying.  I'm 
inclined to start lobbying MS to remove at least that part of the 
runtime checks by default.  Not that it will do us any good.

Or, perhaps, we should just document the caveats with using file 
descriptors directly in python.  That one has to be careful with their 
use, for example, to not steal them from streams that may be using 
them, not closing them when someone else is, and so on.  Use at your 
own risk.  After all, there are numerous ways to crash python through 
reckless use of code, e.g. by using ctypes.
msg86092 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-04-17 21:19
I thought there was a _PyVerify_Fd() just for that, why couldn't it be
used here too?
(not that I think sparkling _PyVerify_Fd() everywhere in our code base
is reasonable and maintainable, but I didn't make that choice :-))
msg180073 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2013-01-16 05:23
Is this still valid because this seems fixed in latest Python 3.4 tip.
msg180087 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2013-01-16 12:29
Though it may be valid in Python 2.7 still.

On 16 January 2013 10:53, Ramchandra Apte <report@bugs.python.org> wrote:

>
> Ramchandra Apte added the comment:
>
> Is this still valid because this seems fixed in latest Python 3.4 tip.
>
> ----------
> nosy: +ramchandra.apte
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue5773>
> _______________________________________
>
msg188828 - (view) Author: Anselm Kruis (anselm.kruis) * Date: 2013-05-10 12:05
Hi,

I was faced with a very similar problem also caused by an invalid file descriptor.

My solution is to set an invalid parameter handler, that does nothing. This effectively disables Dr. Watson. Perhaps this is a suitable solution for other users too. And it does not require a patch.

def set_invalid_parameter_handler(flag):
    """
    Set the MSVCRT invalid parameter handler.
    
    If flag is True, this function sets an invalid parameter handler,
    that does nothing. This effectively disables Dr. Watson.
    If flag is an integer number, it must be the address of an 
    invalid parameter handler function. 
    If flag is None, this function removes the invalid parameter
    handler. This effectively enables Dr. Watson.
    
    The return value is the address of the current handler or None,
    if no handler is installed.
    
    Example::
    
        old = set_invalid_parameter_handler(True)
        try:
            do_something_nasty
        finally:
            set_invalid_parameter_handler(old)
     
    """
    try:
        # get the msvcrt library
        import ctypes.util
        libc = ctypes.util.find_msvcrt()
        if not libc:
            # probably not windows
            return None
        libc = getattr(ctypes.cdll, libc)
        siph = libc._set_invalid_parameter_handler
        siph.restype = ctypes.c_void_p
        siph.argtypes = [ ctypes.c_void_p ]
        # now we need a suitable handler. 
        # The handler must simply return without performing any actions.
        # Of course there is none.
        # But if we look at the calling convention (cdecl), and 
        # at the fact, that we don't need the argument values
        # we find, that we can call any function, as long as the function 
        # does not harm. A suitable function is "int abs(abs)".
        null_handler = libc.abs
    except Exception:
        # probably not the correct windows version 
        return None
    if flag is True:
        flag = null_handler
    return siph(flag)
msg222923 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-13 12:51
Using latest default.

c:\cpython\PCbuild>python_d -c "import os; os.fdopen(2)"
-c:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=2 mode='r' encoding='cp1252'>

Do we need to do anything with 2.7, can this be closed or what?
msg382150 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-11-30 14:23
Python 2.7 is no longer being maintained.
History
Date User Action Args
2020-11-30 14:23:14iritkatrielsetstatus: open -> closed

nosy: + iritkatriel
messages: + msg382150

resolution: out of date
stage: resolved
2019-04-26 19:08:57BreamoreBoysetnosy: - BreamoreBoy
2014-07-13 12:51:42BreamoreBoysetnosy: + BreamoreBoy

messages: + msg222923
versions: - Python 2.6
2013-05-10 12:05:50anselm.kruissetnosy: + anselm.kruis
messages: + msg188828
2013-01-16 12:29:58Ramchandra Aptesetmessages: + msg180087
2013-01-16 05:23:22Ramchandra Aptesetnosy: + Ramchandra Apte
messages: + msg180073
2009-04-17 21:19:07pitrousetnosy: + pitrou
messages: + msg86092
2009-04-17 10:18:09kristjan.jonssonsetmessages: + msg86065
2009-04-16 16:18:45amaury.forgeotdarcsetmessages: + msg86035
2009-04-16 12:51:49kristjan.jonssonsetmessages: + msg86027
2009-04-16 12:39:08amaury.forgeotdarccreate