Under Debian/Lenny/i486, I install python3.0rc3.
After untar the Python3.0rc3 package, I replace in the configure
"/usr/local" by "/usr". After I do :
"configure --with-pydebug --with-doc-strings --enable-shared
--enable-profiling --enable-ipv6 --with-threads --with-tsc ; make"

The result during make is :
"Failed to find the necessary bits to build these modules:
_dbm                                                  
To find the necessary bits, look in setup.py in detect_modules() for the
module's name."

The release of the dbm library is : 
libgdbm-dev   1.8.3-3  GNU dbm database routines (development files)

The header for dbm : "_gdbmmodule.c" request "gdbm.h", and is here
"/usr/include/gdbm.h"

The "Modules/_dbmmodule.c" use :
#include <gdbm/ndbm.h>" ... #include <ndbm.h> ...
but I have only "/usr/include/gdbm-ndbm.h"

Can you patch the "_dbmmodule.c" to use it ?

Patching _dbmmodule.c alone isn't sufficient.  At minimum setup.py would
have to be adjusted to detect the presence of the odd ndbm.h file.

Can you try the attached patch and let us know if it works for you?

Here's a similarly untested patch for Python 2.x.  Can you try it as well?

- I modify your dbm2.diff because the module name is "_dbmmodule.c" and
not "dbmmodule.c". 
- Patching success
- new test "configure ... ; make" and make stop with error :
"...
gcc -pthread -shared
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/_ctypes.o
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/callbacks.o
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/callproc.o
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/stgdict.o
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/cfield.o
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/malloc_closure.o
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/libffi/src/prep_cif.o
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/libffi/src/x86/ffi.o
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_ctypes/libffi/src/x86/sysv.o
-L/usr/local/lib -L. -lpython3.0 -o
build/lib.linux-i686-3.0-pydebug/_ctypes.so

Failed to build these modules:
_dbm                                                  

running build_scripts
creating build/scripts-3.0
copying and adjusting /install/python-3.0rc3.diff/Tools/scripts/pydoc ->
build/scripts-3.0
copying and adjusting /install/python-3.0rc3.diff/Tools/scripts/idle ->
build/scripts-3.0
copying and adjusting /install/python-3.0rc3.diff/Tools/scripts/2to3 ->
build/scripts-3.0
copying and adjusting /install/python-3.0rc3.diff/Lib/smtpd.py ->
build/scripts-3.0
changing mode of build/scripts-3.0/pydoc from 644 to 755
changing mode of build/scripts-3.0/idle from 644 to 755
changing mode of build/scripts-3.0/2to3 from 644 to 755
changing mode of build/scripts-3.0/smtpd.py from 644 to 755
[88798 refs]"

I retest with light configure modification and dbm2.diff with file name
"_dbmmodule.c" :

"configure --with-pydebug --with-doc-strings --enable-shared
--enable-profiling --enable-ipv6 --with-threads --with-tsc --prefix=/usr
; make"

"...
gcc -pthread -shared
build/temp.linux-i686-3.0-pydebug/install/python-3.0rc3.diff/Modules/_dbmmodule.o
-L/usr/local/lib -L. -lgdbm -lpython3.0 -o
build/lib.linux-i686-3.0-pydebug/_dbm.so
*** WARNING: renaming "_dbm" since importing it failed:
build/lib.linux-i686-3.0-pydebug/_dbm.so: undefined symbol: dbm_firstkey

Failed to build these modules:
_dbm                                                  

running build_scripts
[89618 refs]"

Leger> - I modify your dbm2.diff because the module name is
    Leger>   "_dbmmodule.c" and not "dbmmodule.c".

I think you missed the point of the dbm2.diff file.  It should be applied to
the Python 2.x code, not to Python 3.x.  I don't have access to the
necessary platform to run the test.

To summarize:

    * dbm.diff should be applied to the py3k branch (or to the 3.0rc3 release
      code).
    * dbm2.diff should be applied to the code on the trunk (or to the 2.6
      release code).

Skip

Skip, Sorry for my mistake with dbm2.diff ;)

From the tar 3.0rc3 I apply the patch (dbm.diff). Configure ... and
make. But the result is always wrong :
"Failed to build these modules:
_dbm"

Best regards, Frederic

Frederic> From the tar 3.0rc3 I apply the patch (dbm.diff). Configure
    Frederic> ... and make. But the result is always wrong :

    > "Failed to build these modules:
    > _dbm"

Why does it fail?  Is there compile error?  Does linking fail?  I guessed
that the library would still be "gdbm", but I don't really know.  Does your
system maybe install a library file named "libgdbm-ndbm.so" or something
similarly obscure?

I'm happy to try and get this resolved, but not that there is only another
day or two before 3.0final is released.  (This might not make the release
anyway.)

- Only the make generate last error sentence.
- Debian/Lenny provide only "libgdbm-dev-1.8.3-3" provide :
root@xxx:/install# ll /usr/lib/libgdbm.so*
lrwxrwxrwx 1 root root    16 2008-12-01 21:45 /usr/lib/libgdbm.so ->
libgdbm.so.3.0.0
lrwxrwxrwx 1 root root    16 2008-03-08 21:38 /usr/lib/libgdbm.so.3 ->
libgdbm.so.3.0.0
-rw-r--r-- 1 root root 19940 2006-04-24 05:25 /usr/lib/libgdbm.so.3.0.0

I install the Python3.0 (final), the test make is the same and the
result too. I join the output.

I'm running the same distro as Leger and I was having the same problem.
Now I've applied dbm.diff and with a clean build I'm seeing this:

building '_dbm' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -DHAVE_GDBM_DASH_NDBM_H -I.
-I/home/andy/src/python3/Python-3.0/./Include -I. -IInclude -I../Include
-I/usr/local/include -I/home/andy/src/python3/Python-3.0/debug/Include
-I/home/andy/src/python3/Python-3.0/debug -c
/home/andy/src/python3/Python-3.0/Modules/_dbmmodule.c -o
build/temp.linux-i686-3.0/home/andy/src/python3/Python-3.0/Modules/_dbmmodule.o
gcc -pthread -shared
build/temp.linux-i686-3.0/home/andy/src/python3/Python-3.0/Modules/_dbmmodule.o
-L/usr/local/lib -lgdbm -o build/lib.linux-i686-3.0/_dbm.so
*** WARNING: renaming "_dbm" since importing it failed:
build/lib.linux-i686-3.0/_dbm.so: undefined symbol: dbm_firstkey

In Debian Lenny libgbdm-dev provides two libs, libgdbm and libgdbm_compat:

andy@plato:~$ objdump -t /usr/lib/libgdbm.a | grep dbm_firstkey
00000000         *UND*	00000000 gdbm_firstkey
00000140 g     F .text	00000056 gdbm_firstkey

andy@plato:~$ objdump -t /usr/lib/libgdbm_compat.a | grep dbm_firstkey
00000000         *UND*	00000000 gdbm_firstkey
00000000 g     F .text	00000060 dbm_firstkey
00000000         *UND*	00000000 gdbm_firstkey

So it looks like we need to link against -lgdbm_compat instead of -lgdbm
for the _dbm module. I'm attaching dbm3.diff which seems to fix the
problem (I did a test build with it).

Andrew> I'm running the same distro as Leger and I was having the same
    Andrew> problem.  Now I've applied dbm.diff and with a clean build I'm
    Andrew> seeing this:

    ...

    Andrew> *** WARNING: renaming "_dbm" since importing it failed:
    Andrew> build/lib.linux-i686-3.0/_dbm.so: undefined symbol: dbm_firstkey

It looks like we might need to be linking with -lgdbm_compat as well as
-lgdbm.

Can you attach the output of these commands

    objdump -T /usr/lib/libgdbm.so.N
    objdump -T /usr/lib/libgdbm_compat.so.N

to this issue (where N is the highest version available)?

Thanks,

Here's a new version of the patch for Python 3.0.  It appends gdbm_compat
to the gdbm libs if that's where dbm_firstkey is defined.  Please back
out the previous patch against setup.py and Modules/_dbmmodule.c and
apply this one.

Thanks...

Skip, the new patch makes it fail with (highlights):

...
  File "/home/andy/src/python3/Python-3.0/Lib/distutils/ccompiler.py",
line 844, in has_function
    import tempfile
  File "/home/andy/src/python3/Python-3.0/Lib/tempfile.py", line 35, in
<module>
    from random import Random as _Random
  File "/home/andy/src/python3/Python-3.0/Lib/random.py", line 42, in
<module>
    from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil
as _ceil
ImportError: No module named math
make: *** [sharedmods] Error 1


The has_function source in Lib/distutils/ccompiler.py has this comment:

        # this can't be included at module scope because it tries to
        # import math which might not be available at that point - maybe
        # the necessary logic should just be inlined?

Andrew> ImportError: No module named math
    Andrew> make: *** [sharedmods] Error 1

    Andrew> The has_function source in Lib/distutils/ccompiler.py has this
    Andrew> comment:

    Andrew> # this can't be included at module scope because it tries to
    Andrew> # import math which might not be available at that point - maybe
    Andrew> # the necessary logic should just be inlined?

*sigh* I guess that would explain why it wasn't used anywhere in setup.py.

Can you try this simplified version?  (It's probably almost exactly what you
came up with.)

Thanks,

Skip

Skip, your simplified patch works for me. Makes it build fine with the
following snippet:

building '_dbm' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -DHAVE_GDBM_DASH_NDBM_H -I.
-I/home/andy/src/python3/Python-3.0/./Include -I. -IInclude -I../Include
-I/usr/local/include -I/home/andy/src/python3/Python-3.0/debug/Include
-I/home/andy/src/python3/Python-3.0/debug -c
/home/andy/src/python3/Python-3.0/Modules/_dbmmodule.c -o
build/temp.linux-i686-3.0/home/andy/src/python3/Python-3.0/Modules/_dbmmodule.o
gcc -pthread -shared
build/temp.linux-i686-3.0/home/andy/src/python3/Python-3.0/Modules/_dbmmodule.o
-L/usr/local/lib -lgdbm -lgdbm_compat -o build/lib.linux-i686-3.0/_dbm.so

Thanks.

This patch resolve the problem. Evidence with "make test" :
...
test_dbm
test_dbm_dumb
test_dbm_gnu
test_dbm_ndbm
...

Many thanks Skip. You can close this bug I think.

Hi, I'd like to confirm that Skip's last patch fixes the issue.  Hope it
gets included soon!

Thanks

Fix checked in for py3k (r67612), release26-maint (r67613),
release30-maint (r67615) branches as well as trunk (r67614).

now 2008-12-06 trunk fail on linux

may be patch is not applied correctly :
http://svn.python.org/view/python/trunk/setup.py?rev=67614&view=auto

        libraries = gdbm_libs ) )
?   exts.append( Extension('dbm', ['dbmmodule.c'],
?     define_macros=[('HAVE_GDBM_NDBM_H',None)],
?     libraries = ['gdbm'] ) )
  elif db_incs is not None:

Roumen> may be patch is not applied correctly :
    Roumen> http://svn.python.org/view/python/trunk/setup.py?rev=67614&view=auto

    Roumen>         libraries = gdbm_libs ) )
    Roumen> ?   exts.append( Extension('dbm', ['dbmmodule.c'],
    Roumen> ?     define_macros=[('HAVE_GDBM_NDBM_H',None)],
    Roumen> ?     libraries = ['gdbm'] ) )
    Roumen>   elif db_incs is not None:

Yeah, muffed it.  I should have a fix checked in shortly.

Skip

I'm not sure that recent commits in trunk are correct.

Please confirm that build of dbm module with "Berkeley DB" is deprecated.

Note that exist another module gdbm[gdbmmodule.c].

Roumen> I'm not sure that recent commits in trunk are correct.

    Roumen> Please confirm that build of dbm module with "Berkeley DB" is
    Roumen> deprecated.

Can you explain how you think they are incorrect?  The code to decide how to
build the dbm module looks like this:

    if we have ndbm.h:
        build dbm using ndbm
    else if we have a gdbm library:
        if we have gdbm/ndbm.h:
            build dbm using gdbm
        else if we have gdbm-ndbm.h:
            build dbm using gdbm
    else if we found Berkeley includes:
        build dbm using Berkeley db.

This overall structure agrees with that in the Python 2.4 and 2.5 setup.py.
The only thing that changed (in theory) was that there's an extra check for
gdbm-ndbm.h after we've already decided to build dbm with gdbm.  The
preference is still ndbm before gdbm before BerkDB.  That's not to say it's
correct, but nothing's changed as far as I can tell.

Maybe it should look more like this:

    dbmext = None
    if we have ndbm.h:
        ...
        dbmext = Extension(...)

    if dbmext is None and we have a gdbm library:
        if we have gdbm/ndbm.h:
            ...
            dbmext = Extension(...)
        else if we have gdbm-ndbm.h:
            ...
            dbmext = Extension(...)

    if dbmext is None and we found Berkeley includes:
        ...
        dbmext = Extension(...)

    if dbmext is not None:
        exts.append(dbmext)

That still leaves the order ndbm before gdbm before BerkDB.

Skip

The old if statement was "flat":
if find_file("ndbm.h", inc_dirs, []) is not None:
   <CODE>
elif (self.compiler.find_library_file(lib_dirs, 'gdbm')
      and find_file("gdbm/ndbm.h", inc_dirs, []) is not None):
   <CODE>
elif db_incs is not None:
   <CODE>
else:
   miss
If the second case fail, try third and if  succeed build dbm with
"berkeley db".

The new if statement contain nested if:
Now the second case is only "self.compiler.find_library_file(lib_dirs,
'gdbm'):" and if succeed (my case) =>
Failed to find the necessary bits to build these modules:
... dbm ...
Note the build system lack headers "gdbm/ndbm.h", "gdbm-ndbm.h".


To restore previous I add copy of code from third case as new case in
nested if:
===============
                         'dbm', ['dbmmodule.c'],
                         define_macros=[('HAVE_GDBM_DASH_NDBM_H',None)],
                         libraries = gdbm_libs ) )
+                elif db_incs is not None:
+                    exts.append( Extension('dbm', ['dbmmodule.c'],
+                        library_dirs=dblib_dir,
+                        runtime_library_dirs=dblib_dir,
+                        include_dirs=db_incs,
+                        define_macros=[('HAVE_BERKDB_H',None),
+                                       ('DB_DBM_HSEARCH',None)],
+                        libraries=dblibs))
                 else:
                     missing.append('dbm')
             elif db_incs is not None:
===============
Note that above is not proposed patch as I don't like nested ifs and
code duplicate.
Is possible the checks to be in a new separate function ?

Roumen> The old if statement was "flat":
    ...

Okay, I understand now.

    Roumen> The new if statement contain nested if:
    Roumen> Now the second case is only "self.compiler.find_library_file(lib_dirs,
    Roumen> 'gdbm'):" and if succeed (my case) =>
    Roumen> Failed to find the necessary bits to build these modules:
    Roumen> ... dbm ...
    Roumen> Note the build system lack headers "gdbm/ndbm.h", "gdbm-ndbm.h".

In my mind that simply means we haven't accounted for how your system has
organized its header files.  It has a libgdbm so it probably has some sort
of gdbm header file unless you have failed to install some package.

I'm beginning to think this area needs more work.  Let's leave this ticket
closed.  I'll open a new one so we can figure out the best way to tackle
this.  In the meantime can you figure out why your system has a gdbm library
but apparently no gdbm headers (or different ones than we are currently
testing for)?

Thx,

Skip

Skip> I'm beginning to think this area needs more work.  Let's leave
    Skip> this ticket closed.  I'll open a new one so we can figure out the
    Skip> best way to tackle this.

http://bugs.python.org/issue4587

Skip

This is still an issue in Python 2.7.3 but there is a quick manual workaround. I know it's trivial and one can easily develop it from what is said in the thread or maybe looking at the patches, but for reference this is a nice recipe as oppose to digging through many messages

1) build as usual, but redirect the output/error stream in a file, for example if your shell is bash (I find this to always be a good idea):

make > make.log 2>&1

2) Towards the end of make.log there will be the following message. If you don't see this message, you don't have this problem, but possibly a different one:

Failed to build these modules:
dbm


3) execute
grep "\-o .*/dbmmodule.o" make.log

This will find a compiler line. Cut, paste and and execute that command

4) grep "\-o .*/dbm.so" make.log

This will find another compiler line. Cut, paste and and execute that command, ADDING (this is essential) -lgdbm_compat

5) (optional) you may want to remove dbm_failed.so (in the same directory where the previous bullet creates dbm.so)