This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: cygwin builds do not embed
Type: Stage:
Components: Build Versions: Python 2.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jlt63 Nosy List: dradul, jdsumsion, jlt63
Priority: normal Keywords:

Created on 2003-08-24 15:18 by dradul, last changed 2022-04-10 16:10 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
pyport.h.diff jlt63, 2003-08-26 13:46
embed.diff jlt63, 2003-09-02 20:11
embed2.diff jlt63, 2003-09-04 11:37
Messages (16)
msg17922 - (view) Author: Pedro A. Lopez-Valencia (dradul) Date: 2003-08-24 15:18
As an example case, try to build a copy of vim with an 
embeded python interpreter:

"""
indow.o   objects/if_python.o objects/py_config.o     
objects/netbeans.o  object
s/version.o       -lncurses   -liconv -lintl   -
L/usr/lib/python2.3/config -lpyt
hon2.3 -lutil -lm
objects/py_config.o(.data+0x4):config.c: referencia a 
`_initthread' sin definir
objects/py_config.o(.data+0xc):config.c: referencia a 
`_initsignal' sin definir
objects/py_config.o(.data+0x14):config.c: referencia a 
`_initposix' sin definir
objects/py_config.o(.data+0x1c):config.c: referencia a 
`_initerrno' sin definir
objects/py_config.o(.data+0x24):config.c: referencia a 
`_init_sre' sin definir
objects/py_config.o(.data+0x2c):config.c: referencia a 
`_init_codecs' sin defini
r
"""

As you can see, it doesn't embed. 

I have tracked down the failure to a problem in the 
LDSHARED defaults provided by configure.in. The default 
supplied is:

LDSHARED="gcc -shared -Wl,--enable-auto-image-
base";;

but this *does not* work. As per Cygwin documentation 
on the creation of DLLs, this works (already tested 
compiling vim under the same conditions):

LDSHARED="gcc -shared -Wl,--enable-auto-image-base -
Wl,--export-all-symbols -Wl,--enable-auto-import";;

As a side note, please notice that this problem, proper 
user of GNU ld flags under win32, has a direct impact on 
the possible success of a true Mingw32 port (which I 
expect eagerly).
msg17923 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-08-25 18:32
Logged In: YES 
user_id=86216

AFAICT, changing LDSHARED will affect building extensions
too. So, I am very hesitant to change its definition without
fully understanding the ramifications. Note that Cygwin
Python extensions have been working for almost three
years now. Additionally, I WAG that there are many more
people building extensions than embedding.

> LDSHARED="gcc -shared -Wl,--enable-auto-image-base
> -Wl,--export-all-symbols -Wl,--enable-auto-import";;

I can understand why adding "-Wl,--export-all-symbols"
fixes the above link errors. However, I don't understand
why "-Wl,--enable-auto-import" would help. IIRC, it
defaults to enabled anyway. Does vim with an 
embeded python interpreter build without this option?

Can you provide me a small embedded example so I
can do some testing? Using vim seems a bit unwieldy.

> As a side note, please notice that this problem, proper 
> user of GNU ld flags under win32, has a direct impact on 
> the possible success of a true Mingw32 port (which I 
> expect eagerly).

Huh? How is this related to the report Cygwin problem?
msg17924 - (view) Author: Pedro A. Lopez-Valencia (dradul) Date: 2003-08-26 10:57
Logged In: YES 
user_id=659006

Ramifications to changing LDSHARED? I don't perceive any 
besides having slightly bigger dlls, and the possibility to 
extract symbol definitions and create import libraries out of 
plugins, perhaps even linking extensoins directly (as in writing 
derivative extensions from, say, Numeric).

Adding -Wl,--enable-auto-import doesn't modify the defaults 
but hushes the linker, less perturbing line noise in stderr.

On giving you an example, I only use python under cygwin 
and having just managed to build my first embedded vim, I'm 
starting to experiment. There are some reasonable examples 
in vim's documentation (see ':h if_pyth.txt'). This works for 
me:

1. Type a line of text. Leave cursor on line.
2. Enter command mode and type:
    python import vim; vim.current.line = vim.current.line.upper
()

And about your "Huh?", this is the python bug tracker last 
time I checked. The issue is a GNU ld/Win32 issue, not 
necessarily exclusive to Cygwin. I am leaving a hint for future 
developers.

Cheers.
msg17925 - (view) Author: Pedro A. Lopez-Valencia (dradul) Date: 2003-08-26 11:00
Logged In: YES 
user_id=659006

Ramifications to changing LDSHARED? I don't perceive any 
besides having slightly bigger dlls, and the possibility to 
extract symbol definitions and create import libraries out of 
plugins, perhaps even linking extensoins directly (as in writing 
derivative extensions from, say, Numeric).

Adding -Wl,--enable-auto-import doesn't modify the defaults 
but hushes the linker, less perturbing line noise in stderr.

On giving you an example, I only use python under cygwin 
and having just managed to build my first embedded vim, I'm 
starting to experiment. There are some reasonable examples 
in vim's documentation (see ':h if_pyth.txt'). This works for 
me:

1. Type a line of text. Leave cursor on line.
2. Enter command mode and type:
    python import vim; vim.current.line = vim.current.line.upper
()

And about your "Huh?", this is the python bug tracker last 
time I checked. The issue is a GNU ld/Win32 issue, not 
necessarily exclusive to Cygwin. I am leaving a hint for future 
developers.

Cheers.
msg17926 - (view) Author: Pedro A. Lopez-Valencia (dradul) Date: 2003-08-26 11:05
Logged In: YES 
user_id=659006

Opps!

Sorry, my link fell down in the middle of submitting the form. 
msg17927 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-08-26 13:46
Logged In: YES 
user_id=86216

I would prefer to just export the missing symbols than
all of them which is unnecessary and possibly problematic.

The attached patch (against Python CVS) exports the
following additionally symbols:

$ objdump -p libpython2.3.dll | fgrep '] init'
        [ 794] init_codecs
        [ 795] init_sre
        [ 796] init_symtable
        [ 797] initerrno
        [ 798] initposix
        [ 799] initsignal
        [ 800] initthread
        [ 801] initxxsubtype
        [ 802] initzipimport

Does the above meet your needs? Are you willing to test
your vim build against it?
msg17928 - (view) Author: Pedro A. Lopez-Valencia (dradul) Date: 2003-08-27 13:06
Logged In: YES 
user_id=659006

Yes, I am willing to test it.

Yet... Don't you think you are complicating the issue, the
autotool files and your life, too much? There are no obvious
security issues involved and the procedure only adds extra
text to the DATA segment in the DLLs, there are no changes
in the actual object code; it won't become a hybrid
DLL/implib such as cygwin1.dll! Take any Unix shared library
(Linux, Solaris, BSD, whatever), and it is already exporting
*all symbols*, how else would you be able to link to them?
There are no import libs, that's a Windows/MacOS Classic
abomination as far as I am concerned :-)

Anyway, it's your ballgame, it's your call.

Cheers
msg17929 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-08-27 14:12
Logged In: YES 
user_id=86216

> Yes, I am willing to test it.

Thanks. Are you willing to build against Python CVS? Or,
do you need me to provide you with replacements for
libpython2.3.dll.a and libpython2.3.dll?

> Yet... Don't you think you are complicating the issue, the
> autotool files and your life, too much?

I don't think so. Note that my approach does not affect
any autotools files.

> There are no obvious security issues involved and...

IIRC, exporting all symbols under Win32 can cause
multiple definition link errors.

> Anyway, it's your ballgame, it's your call.

I appreciate your willingness to acquiesce.
msg17930 - (view) Author: Pedro A. Lopez-Valencia (dradul) Date: 2003-08-27 15:13
Logged In: YES 
user_id=659006

I have tested it against the release23-maint CVS branch, and
unfortunately it doesn't work.. 

*Sigh* I mentioned the autotools because I didn't see the
patch at first (the wonders of top posting, who designed
this bug-tracker?). But now... I think we need an export
symbols definition file... :-(
msg17931 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-08-27 15:31
Logged In: YES 
user_id=86216

What do you get when you execute the following?

$ objdump -p libpython2.3.dll | fgrep '] init'

Are you building vim against the newly built Python?
msg17932 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-09-02 20:11
Logged In: YES 
user_id=86216

Please try again with the second patch applied too.  The
combination of them seem to work for me.

> I have tested it against the release23-maint CVS branch,
>and unfortunately it doesn't work.. 

BTW, it would have been helpful it you indicated that the
first patch eliminated all but 3 link errors.
msg17933 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-09-04 11:37
Logged In: YES 
user_id=86216

Combined the original two patches into one and refreshed
against current CVS.
msg17934 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-09-04 11:51
Logged In: YES 
user_id=86216

I should have mentioned that the patch assumes Python's
builtin modules are only written in C (not C++). If this
assumption is incorrect, then I will add the standard
__cplusplus/extern "C" idiom.
msg17935 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-09-04 12:01
Logged In: YES 
user_id=86216

> python import vim; vim.current.line = vim.current.line.upper
()

The above example works fine with embed2.diff applied.

I tested (builing Python) under Red Hat Linux 8.0 too.

Commited as:
Include/pyport.h 2.63
Modules/gcmodule.c 2.72
Python/import.c 2.223
Python/marshal.c 1.75
msg17936 - (view) Author: John Sumsion (jdsumsion) Date: 2003-09-22 19:07
Logged In: YES 
user_id=867854

I'm a newbie at building O/S software, but at least I figured 
this one out (running Cygwin 1.5.5-1 on XP Pro).  Thanks to 
Jason who provided the embed2 patch and helped me 
understand this more fully.

Running on cygwin, if you get the python package (2.3-2), 
follow these steps to get an updated libpython2.3.dll.a 
(and .dll):

cd /usr/src/python-2.3-2
patch -p0 < embed2.diff (attached to this issue, don't use 
embed.diff or pyport.h.diff)
./configure
make
make install (only if you want to install these files)

Now look in the current directory for the .dll.a and the .dll.  I 
was able to build vim 6.2.98 (src also from cygwin dist) with 
an embedded python interpreter with the resulting files (I 
copied the .dll.a to /usr/lib/python2.3/config and the .dll 
to /usr/bin).
msg17937 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2003-10-13 15:45
Logged In: YES 
user_id=86216

The Cygwin python-2.3.2-1 package has this patch already
applied:

http://cygwin.com/ml/cygwin-announce/2003-
10/msg00010.html
History
Date User Action Args
2022-04-10 16:10:49adminsetgithub: 39119
2003-08-24 15:18:03dradulcreate