This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: os.fdopen reopening a read-only fd on windows
Type: Stage: resolved
Components: IO, Windows Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, martin.panter, mattip, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2015-03-10 22:41 by mattip, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
test_openfd.py mattip, 2015-03-10 22:41
Messages (3)
msg237818 - (view) Author: mattip (mattip) * Date: 2015-03-10 22:41
If I have a read-only fd, and I try to open it as 'w' with 

os.fdopen(fd, 'w'), 

the operation raises on linux but succeeds on windows. Python relies on the underlying clib's fdopen to return an error, which glibc does, but MSVC happily closes the original fd and returns with no error.

Would a patch to fix this be welcomed? A test demonstrating the issue is attached. The fix would be to check that the mode requested matches the mode of the argument fd.

Related issues #22259 and #21191 did not consider this case
msg237825 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-03-10 23:35
I guess you were mainly testing with Python 2. Python 3 on Linux does not raise any error either:

wrote 3 to a read-only file
should raise, opening a ro descriptor for writing
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

BTW in your test script you close the underlying file descriptor “fd” before giving the file object “fd2” a chance to flush or close, which is probably why you see no error. Though in Wine I see this, probably because file.write() does not return a value in Python 2:

properly raised TypeError('%d format: a number is required, not NoneType',)

What would be the reason for doing a mode check in fdopen()? Just to detect programmer errors if the wrong mode or file descriptor is given?
msg237845 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-03-11 06:13
> I guess you were mainly testing with Python 2. Python 3 on Linux 
> does not raise any error either

In Python 3 os.fdopen delegates to io.open, which calls io.FileIO to create the raw file object. This doesn't verify a compatible mode on the file descriptor. Similarly, in Windows Python 2 os.fdopen calls VC++ _fdopen, which also doesn't verify a compatible mode. 

The POSIX spec (IEEE Std 1003.1, 2013 Edition) for fdopen says that "the *application* shall ensure that the mode of the stream as expressed by the mode argument is allowed by the file access mode of the open file description to which fildes refers" [1] (emphasis mine).  It happens that glibc in Linux opts to do this check for you.

If instead of closing the underlying file descriptor the program opts to close the Python file object or C FILE stream, this will attempt to write the buffered string "bbb" to the read-only fd, which should raise an EBADF error. 

[1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/fdopen.html

---

The way test_openfd.py directly closes the underlying file descriptor does highlight an annoying problem in Python 2 on Windows. This should get its own issue in case someone feels like addressing it. The problem is open() and os.fdopen() create the file object with fclose as the FILE stream closer. It'd be nicer to instead use a closer on Windows that first calls _PyVerify_fd to check for a valid file descriptor. Otherwise the CRT asserts and terminates the process. For example:

    import os
    f = open('@test')
    os.close(f.fileno())

Functions in posixmodule.c are good about first verifying the file descriptor:

    >>> os.lseek(f.fileno(), 0, os.SEEK_CUR) # calls _PyVerify_fd
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 9] Bad file descriptor

OTOH, closing the file object doesn't verify the file descriptor:

    >>> f.close() # Goodbye cruel world... :'(

Stack trace:

    0:000> k 8
    Child-SP          RetAddr           Call Site
    00000000`0021f3a8 00000000`68e25200 kernel32!TerminateProcessStub
    00000000`0021f3b0 00000000`68e252d4 MSVCR90!invoke_watson+0x11c
    00000000`0021f9a0 00000000`68e1de7e MSVCR90!invalid_parameter+0x70
    00000000`0021f9e0 00000000`68ddf904 MSVCR90!close+0x9e
    00000000`0021fa30 00000000`68ddf997 MSVCR90!fclose_nolock+0x5c
    00000000`0021fa70 00000000`1e0ac2e5 MSVCR90!fclose+0x5f
    00000000`0021fab0 00000000`1e0ac7b2 python27!close_the_file+0xa5
    00000000`0021fae0 00000000`1e114427 python27!file_close+0x12

The problem doesn't exist in Python 3, for which io.FileIO's internal_close function is gated by _PyVerify_fd.
History
Date User Action Args
2022-04-11 14:58:13adminsetgithub: 67822
2021-02-24 11:22:52eryksunsetstatus: open -> closed
resolution: out of date
stage: resolved
2015-03-11 06:13:01eryksunsetnosy: + eryksun

messages: + msg237845
versions: + Python 2.7
2015-03-10 23:35:39martin.pantersetnosy: + martin.panter
messages: + msg237825
components: + IO
2015-03-10 22:41:49mattipcreate