classification
Title: Module 'parser' fails to build
Type: compile error Stage:
Components: Extension Modules Versions: Python 3.0, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: LambertDW, kirkshorts, loewis, msapiro, yselkowitz
Priority: normal Keywords: patch

Created on 2008-11-07 13:58 by kirkshorts, last changed 2009-01-11 09:33 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
nu_diff.txt kirkshorts, 2008-11-09 21:51 Patch to use correct grammar file.
2.6.1-parsermodule.patch yselkowitz, 2009-01-11 01:22 _PyParser_Grammar export patch (take 2)
Messages (13)
msg75603 - (view) Author: Andy (kirkshorts) Date: 2008-11-07 13:58
With a clean checkout of the py3k source it fails to build the Parser 
exension module (based on Modules/parsermodule.c) when building against 
a cygwin target.

This appears to be related to the movement of the symbol 
_PyParser_Grammar from graminit.c to Parser\metagrammar.c. (At least 
this is where it was going by the code comments)

Because of this the module now requires Parser\metagrammar.c to get 
this information via Py_meta_grammar.

The patch modifies setup.py to add the required file and modifies 
parsermodule.c to use the accessor function.

(This fails on a clean trunk build in the same way as well - which 
makes me very suspicious that this is not a "real" issue as the 
buildbots seem to be green.)

My gut feeling is that my modification to setup.py for the module is 
incorrect - it just looks messy. So I await the inevitable: "That's not 
how to fix it...." :-)
msg75664 - (view) Author: Andy (kirkshorts) Date: 2008-11-09 19:58
bah I *am* a idiot, #4288 and Christian's comments point out that I
can't use 'find' & 'xargs' properly :-(

Will modify patch to use the correct grammar file &c.

(and maybe one day I might actually say something sensible to do with
Python development :-) )
msg75665 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-09 20:06
As Christian said in #4288: this links in a separate of metagrammar.c,
which is undesirable. However, I think you can fix this by exporting
Py_meta_grammar from pythonxy.dll.
msg75666 - (view) Author: Andy (kirkshorts) Date: 2008-11-09 21:51
a new patch that will use the grammar definition from Python/graminit.c
- it is as of yet untested for Cygwin (can't get to that machine right
now). It follows the same pattern as the previous, i.e. it makes us of
an accessor function to get the grammar definition.

This has expanded the patch somewhat to include changes to:

 - setup.py for Cygwin environment.
 - Parser/pgenmain.c to write out the function body
 - Python/pythonrun.c to use the new function
 - Include/grammar.h to declare the new function
 - Modules/parsermodule.c to use the new function


All of which makes me think that the change to make the symbol "public"
and use it directly without hiding it is a better way to go.

Will look at this under my Cygwin environment tomorrow. (I have run a
configre - make - test cycle on Ubuntu (hardy heron) and it is OK [but
then its not broken there anyway :-) ] )
msg75667 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-09 22:10
That patch is too complicated. We already have meta_grammar and
Py_meta_grammar, and now you also add a third function
(get_PyParserGrammar) that does the same thing again. I don't see why
you can't call one of the existing functions, and I fail to see the need
to change pythonrun.c at all.
msg75668 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-09 22:12
As a style guide remark: drop the parentheses around the expression in
the return statement (return is a statement, not a function), and prefix
all global symbols with Py or _Py. See PEP 7 for further instructions.
msg75685 - (view) Author: Andy (kirkshorts) Date: 2008-11-10 09:29
Martin:

Looking at it I agree with you 100% - the patch is too complicated for 
what it is intending to resolve. It simply does not need another 
accessor function to muddy the waters when making the symbol public as 
done in #4288 resolves the issue.

My personal preference is to try to hide such data structures - but 
that doesn't always mean its the correct thing to do :-)
msg76445 - (view) Author: Yaakov (Cygwin Ports) (yselkowitz) Date: 2008-11-26 03:23
I can confirm this problem in 3.0rc3 on Cygwin.  If I could get some
direction from the Python devs on which method would be preferred
(accessor function vs. exposing data structure), I would be happy to
provide a patch.
msg76449 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-26 06:52
I think the parser module should call one of the existing functions.
msg76450 - (view) Author: Yaakov (Cygwin Ports) (yselkowitz) Date: 2008-11-26 08:23
Attempting to export and use Py_meta_grammar() gives me a broken module,
based on the results of test/test_parser.py.

This patch, which exports _PyParser_Grammar, is very simple and the test
passes.

I will gladly take guidance on this patch.
msg79584 - (view) Author: Mark Sapiro (msapiro) Date: 2009-01-10 22:53
This problem also occurs when building the 2.6.1 parser module on Cygwin
1.5.25. It did not occur with Python 2.6 or 2.5.x.

The error from 'make' is

building 'parser' extension
gcc -shared -Wl,--enable-auto-image-base
build/temp.cygwin-1.5.25-i686-2.6/cygdrive/c/Python_dist/Python-2.6.1/Modules/parsermodule.o
-L/usr/local/lib -L. -lpython2.6 -o
build/lib.cygwin-1.5.25-i686-2.6/parser.dll
build/temp.cygwin-1.5.25-i686-2.6/cygdrive/c/Python_dist/Python-2.6.1/Modules/parsermodule.o:
In function `parser_expr':
/cygdrive/c/Python_dist/Python-2.6.1/Modules/parsermodule.c:552:
undefined reference to `__PyParser_Grammar'
build/temp.cygwin-1.5.25-i686-2.6/cygdrive/c/Python_dist/Python-2.6.1/Modules/parsermodule.o:
In function `parser_suite':
/cygdrive/c/Python_dist/Python-2.6.1/Modules/parsermodule.c:552:
undefined reference to `__PyParser_Grammar'
collect2: ld returned 1 exit status

I was able to work around the error and build a parser module that
passed unit test by manually running

gcc -shared -Wl,--enable-auto-image-base
build/temp.cygwin-1.5.25-i686-2.6/cygdrive/c/Python_dist/Python-2.6.1/Modules/parsermodule.o
Python/graminit.o -L/usr/local/lib -L. -lpython2.6 -o
build/lib.cygwin-1.5.25-i686-2.6/parser.dll

i.e. by including Python/graminit.o in the explicit object files to load.

I have also confirmed that applying the parser-grammar.patch from #4288
will allow make to successfully build a parser module that passes unit
tests.
msg79585 - (view) Author: Yaakov (Cygwin Ports) (yselkowitz) Date: 2009-01-11 01:22
Here's the patch I used for the Cygwin Ports 2.6 and 3.0 packages.  It's
the simplest working solution that I could find.
msg79592 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-01-11 09:33
I have now committed 2.6.1-parsermodule.patch as r68523, r68524, r68525,
and r68526. Thanks for the patch.
History
Date User Action Args
2009-01-11 09:35:19loewislinkissue4288 superseder
2009-01-11 09:33:52loewissetstatus: open -> closed
resolution: fixed
messages: + msg79592
2009-01-11 01:22:55yselkowitzsetfiles: + 2.6.1-parsermodule.patch
messages: + msg79585
2009-01-10 22:53:39msapirosetnosy: + msapiro
messages: + msg79584
versions: + Python 2.6
2008-11-26 09:24:55yselkowitzsetfiles: - 3.0rc3-parsermodule.patch
2008-11-26 08:23:36yselkowitzsetfiles: + 3.0rc3-parsermodule.patch
messages: + msg76450
2008-11-26 06:52:43loewissetmessages: + msg76449
2008-11-26 03:23:14yselkowitzsetnosy: + yselkowitz
messages: + msg76445
2008-11-10 09:29:15kirkshortssetmessages: + msg75685
2008-11-09 22:12:41loewissetmessages: + msg75668
2008-11-09 22:10:03loewissetmessages: + msg75667
2008-11-09 21:51:28kirkshortssetfiles: - parsermodule_fix.diff
2008-11-09 21:51:19kirkshortssetfiles: + nu_diff.txt
messages: + msg75666
2008-11-09 20:06:36loewissetnosy: + loewis
messages: + msg75665
2008-11-09 19:58:25kirkshortssetmessages: + msg75664
2008-11-07 14:58:01LambertDWsetnosy: + LambertDW
2008-11-07 13:58:04kirkshortscreate