classification
Title: On Windows, APIs passing FILE* break with Borland C (PR#121)
Type: enhancement Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution: later
Dependencies: Superseder:
Assigned To: jhylton Nosy List: gvanrossum, jhylton, nobody, strife35
Priority: normal Keywords:

Created on 2000-08-01 21:10 by anonymous, last changed 2001-08-23 16:36 by gvanrossum. This issue is now closed.

Messages (11)
msg670 - (view) Author: Nobody/Anonymous (nobody) Date: 2000-08-01 21:10
Jitterbug-Id: 121
Submitted-By: "Brad Clements" <bkc@murkworks.com>
Date: Wed, 3 Nov 1999 15:53:01 -0500
Version: None
OS: None

The system encountered a fatal error 
After command: 
Received: 
The last error code was: Connection refused 

Web interface using {HYPERLINK "http://samba.anu.edu.au/jitterbug/"}Jitterbug 
{HYPERLINK "../python-bugs/"}Public interface 
{HYPERLINK "../python-bugs.private"}Private interface (requires password) 
{HYPERLINK "http://www.python.org/"}Python home page 

-----
Here's what I said for Python 1.5.2 on Win32 #0

PyRun_SimpleFile, PyRun_File and other functions that take a FILE * are not 
usable on WIN32 from non-VC applications because python15.dll is statically 
linked to the MS runtime DLL. Embedding applications that try to use these 
functions are passing in FILE * structures that do not match MS's runtime 
format. 

For example, I'm using Python in a Borland C++ Builder application. Although I 
can open a FILE *, when passed to python15.dll the FILE * is not usable.

The addition of two helper functions would solve this problem:

FILE * PyRun_OpenFile(char *file, char *mode) 
{
  return fopen(file,mode)
}

int PyRun_CloseFile(FILE *ptr)
{
  return fclose(ptr)
}

This way embedding apps could get python15.dll to open the file and it would 
work.

A temporary workaround is to always load the .pyc file in PyRun_SimpleFile..


Brad Clements,                bkc@murkworks.com   (315)268-1000
http://www.murkworks.com                          (315)268-9812 Fax
netmeeting: ils://ils.murkworks.com               ICQ: 14856937


====================================================================
Audit trail:
Fri Nov 05 10:31:10 1999	guido	moved from incoming to request
msg671 - (view) Author: Nobody/Anonymous (nobody) Date: 2000-08-01 21:10
From: Guido van Rossum <guido@CNRI.Reston.VA.US>
Subject: Re: [Python-bugs-list] Python 1.5.2 bug, tried to post but got error (PR#121)
Date: Wed, 03 Nov 1999 20:08:11 -0500

> I tried to post a bug, but got this error:
> 
> The system encountered a fatal error 

We were being slammed by a defective spider (or, if you're more
paranoid, by a hacker) so we temporarily turned off the webserver.  It
should be back on now.

> Here's what I said for Python 1.5.2 on Win32 #0
> 
> PyRun_SimpleFile, PyRun_File and other functions that take a FILE * are not 
> usable on WIN32 from non-VC applications because python15.dll is statically 
> linked to the MS runtime DLL. Embedding applications that try to use these 
> functions are passing in FILE * structures that do not match MS's runtime 
> format. 
> 
> For example, I'm using Python in a Borland C++ Builder application. Although I 
> can open a FILE *, when passed to python15.dll the FILE * is not usable.
> 
> The addition of two helper functions would solve this problem:
> 
> FILE * PyRun_OpenFile(char *file, char *mode) 
> {
>   return fopen(file,mode)
> }
> 
> int PyRun_CloseFile(FILE *ptr)
> {
>   return fclose(ptr)
> }
> 
> This way embedding apps could get python15.dll to open the file and it would 
> work.
> 
> A temporary workaround is to always load the .pyc file in PyRun_SimpleFile..

This is an elegant solution.  I think I'll add it.

Could you mail me your suggestion again with the legal boilerplate
included?  See http://www.python.org/1.5/bugrelease.html for the text
and explanation.  Our lawyers require that I request this silliness...

--Guido van Rossum (home page: http://www.python.org/~guido/)
msg672 - (view) Author: Nobody/Anonymous (nobody) Date: 2000-08-01 21:10
From: "Brad Clements" <bkc@murkworks.com>
Subject: Re: [Python-bugs-list] Python 1.5.2 bug, tried to post but got error (PR#121)
Date: Wed, 3 Nov 1999 20:22:21 -0500

Here it is again.

Also, I believe this will resolve  FAQ entry 8.10


> 8.10. Can't get Py_RunSimpleFile() to work.
> This is very sensitive to the compiler vendor, version and (perhaps) even
> options. If the FILE* structure in your embedding program isn't the same as
> is assumed by the Python interpreter it won't work. The Python 1.5.* DLLs
> (python15.dll) are all compiled with MS VC++ 5.0 and with
> multithreading-DLL options (/MD, I think). 
> 
> If you can't change compilers or flags, try using Py_RunSimpleString(). A
> trick to get it to run an arbitrary file is to construct a call to
> execfile() with the name of your file as argument. 
> 
> 
> 
> ---------------------------------------------------------------------------
> -----


> Here's what I said for Python 1.5.2 on Win32 #0
> 
> PyRun_SimpleFile, PyRun_File and other functions that take a FILE * are not 
> usable on WIN32 from non-VC applications because python15.dll is statically 
> linked to the MS runtime DLL. Embedding applications that try to use these 
> functions are passing in FILE * structures that do not match MS's runtime 
> format. 
> 
> For example, I'm using Python in a Borland C++ Builder application. Although I 
> can open a FILE *, when passed to python15.dll the FILE * is not usable.
> 
> The addition of two helper functions would solve this problem:
> 
> FILE * PyRun_OpenFile(char *file, char *mode) 
> {
>   return fopen(file,mode)
> }
> 
> int PyRun_CloseFile(FILE *ptr)
> {
>   return fclose(ptr)
> }
> 
> This way embedding apps could get python15.dll to open the file and it would 
> work.
> 
> A temporary workaround is to always load the .pyc file in PyRun_SimpleFile..


I confirm that, to the best of my knowledge and belief, this
contribution is free of any claims of third parties under
copyright, patent or other rights or interests ("claims").  To
the extent that I have any such claims, I hereby grant to CNRI a
nonexclusive, irrevocable, royalty-free, worldwide license to
reproduce, distribute, perform and/or display publicly, prepare
derivative versions, and otherwise use this contribution as part
of the Python software and its related documentation, or any
derivative versions thereof, at no cost to CNRI or its licensed
users, and to authorize others to do so.

I acknowledge that CNRI may, at its sole discretion, decide
whether or not to incorporate this contribution in the Python
software and its related documentation.  I further grant CNRI
permission to use my name and other identifying information
provided to CNRI by me for use in connection with the Python
software and its related documentation.



Brad Clements,                bkc@murkworks.com   (315)268-1000
http://www.murkworks.com                          (315)268-9812 Fax
netmeeting: ils://ils.murkworks.com               ICQ: 14856937
msg673 - (view) Author: Jeremy Hylton (jhylton) (Python triager) Date: 2000-09-07 22:01
Please do triage on this bug.
msg674 - (view) Author: Jeremy Hylton (jhylton) (Python triager) Date: 2000-09-12 16:01
Guido -- In 1999, you said you would add this feature.  Did you?
msg675 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2000-09-13 10:53
This is a feature request.
Jeremy, how's that feature request PEP coming?
msg676 - (view) Author: Jeremy Hylton (jhylton) (Python triager) Date: 2000-09-15 19:21
added to pep 42
msg677 - (view) Author: adam edelstein (strife35) Date: 2001-08-23 14:33
Logged In: YES 
user_id=305840

I have a related question, I have a project that I am 
working on, and I ran into an enormous amount of problems 
with file i/o as a result of pythons standard file stuf not 
using win32api, what I'm doing to prevent the problem is re-
building python, and my project (embedded into python) but 
in rebuilding i have edited fileobject.c to force all file 
operation calls to use their win32api counterparts by 
defining macros to replace the stdio functions.  I.E. fopen
() -> createfile(), ...The macro replacement accepts the 
original arguements, converts them as neccessary to the 
ones needed by the win32API func, calls the win32api 
function and then returns a value which models the behavior 
of the original function. 
        As you may have guessed already this presented some 
interesting problems as most of these functions are all 
passing around (FILE *) structures.  To work around this I 
am returning the HANDLE's that these functions return 
letting the program pretend that they are FILE *s.  I would 
have macrod FILE * to be a HANDLE except that one of the 
first functions PyFile_AsFile(f) is of type FILE *, and 
didnt like that very much....     Finally I had replaced 
almost everything..
      Currently I am down to 2 last problems, dealing with 
fputs(), and that in now in my tracebacks sys.stdout 
instead of being an open path to stdout so as to enable 
sys.stdout.write('whatever') to write to stdout and thus to 
screen, it becomes an error as 'my' file opener i believe 
attempts to open a file named stdout..  I havent had much 
luck trying to figure what happens in sysmodule, and was 
wondering if any one had any ideas comments suggestions or 
help to offer.  
msg678 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-08-23 14:38
Logged In: YES 
user_id=6380

To strife35 (Adam Edelstein?):

It might be better to have a separate object type that
resembles a file object in its Python interface but uses the
win32api calls rather than stdio.

Can you explain why you need to use win32api I/O calls
directly rather than via the MS-provided standard C library
(which makes Win32 API calls underneath)?
msg679 - (view) Author: adam edelstein (strife35) Date: 2001-08-23 15:00
Logged In: YES 
user_id=305840

The problem arrises in that this project is running over a 
network, and later I get problems with lockfiles and 
deleting files because although files may appear to 
be "closed" at the python level from the old stdio 
functions like fclose, they are indeed still open at the NT 
level and as a result cant be deleted.  This becomes a real 
problem with OCI files, and other things on NT.  It is not 
a problem on other os's like Solaris, Unix, linux, hpux, 
etc.., it is a WINnt problem
                    Adam
msg680 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-08-23 16:36
Logged In: YES 
user_id=6380

Hm.  Are you *sure* that you are calling the close()
method?  I find it hard to believe that the stdio fclose()
wouldn't make the corresponding Win32 call to really close
the object. A fairly common failure mode in Python is to
have a local variable referencing an open file and assuming
that when the local variable goes out of scope, the file is
closed.  This will *usually* happen, but sometimes the stack
frame is kept alive (e.g. by exception handling machinery)
and that prevents the file from being closed.

I am very familiar with the Windows problem that you can't
remove/rename a file that's still open -- but I've never
heard of a case where making the explicit close() call
doesn't fix it.

Just trying to be helpful -- what you're embarking on seems
a big project...
History
Date User Action Args
2000-08-01 21:10:13anonymouscreate