classification
Title: test_dbm fails when run directly
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: bmoyles, larry, miss-islington, serhiy.storchaka, terry.reedy, vstinner
Priority: normal Keywords: patch

Created on 2017-07-25 12:10 by serhiy.storchaka, last changed 2019-10-29 08:00 by larry. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6632 merged serhiy.storchaka, 2018-04-29 11:49
PR 6634 merged miss-islington, 2018-04-29 12:46
PR 6635 merged miss-islington, 2018-04-29 12:48
Messages (13)
msg299055 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-25 12:10
$ ./python Lib/test/test_dbm.py
..BDB0004 fop_read_meta: @test_29975_tmp_ndbm.db: unexpected file type or format
..EBDB3028 @test_29975_tmp.db: unable to flush: No such file or directory
..EBDB3028 @test_29975_tmp.db: unable to flush: No such file or directory
.E.........
======================================================================
ERROR: test_anydbm_creation (__main__.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_dbm.py", line 73, in test_anydbm_creation
    self.read_helper(f)
  File "Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

======================================================================
ERROR: test_anydbm_modification (__main__.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_dbm.py", line 88, in test_anydbm_modification
    self.read_helper(f)
  File "Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

======================================================================
ERROR: test_anydbm_read (__main__.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_dbm.py", line 94, in test_anydbm_read
    self.read_helper(f)
  File "Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

----------------------------------------------------------------------
Ran 19 tests in 0.683s

FAILED (errors=3)
msg299058 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-25 12:19
On my Linux (Fedora 25), "./python Lib/test/test_dbm.py" and "./python -m test test_dbm" both succeed.
msg299061 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-25 12:22
Which dbm implementation are you using? Do you have the following modules?

haypo@selma$ ./python 
Python 3.7.0a0 (heads/master:3924377, Jul 25 2017, 14:17:41) 
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _gdbm
>>> import _dbm
msg299073 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-25 13:05
>>> import _gdbm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named '_gdbm'
>>> import _dbm
>>>
msg299442 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-07-29 00:14
OK on Windows, skipping test_whichdb_ndbm (neither _dbm nor _gdbm are present.)
msg301740 - (view) Author: Brian Moyles (bmoyles) Date: 2017-09-08 21:43
I ran into the same problem when running the test suite against a fresh Python 3.6.2 build on Ubuntu 16.04. It specifically appears to be a problem with ndbm and retrieving an empty byte string value:

Python 3.6.2 (default, Sep  8 2017, 18:31:28)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbm.gnu, dbm.ndbm
>>> g = dbm.gnu.open('gnudb', 'c')
>>> g['0'.encode('ascii')] = b''
>>> g.keys()
[b'0']
>>> g['0'.encode('ascii')]
b''
>>>
>>> n = dbm.ndbm.open('ndbm', 'c')
>>> n['0'.encode('ascii')] = b''
>>> n.keys()
[b'0']
>>> n['0'.encode('ascii')]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: b'0'

Same behavior when using the distribution's stock Python 3.5.2 so I suspect there's a behavioral change in libdb that's not accounted for?
msg301756 - (view) Author: Brian Moyles (bmoyles) Date: 2017-09-08 23:15
Actually, I was able to make things happy by removing libdb5.3-dev and solely relying on libgdbm3-dev (which suggests that Debian upstream may want to pare back their build requirements rather than disable the dbm test as they appear to do now).
msg315888 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-29 11:48
The same error is now occurred in Ubuntu 18.04 by default. 

The proposed patch moves the test for the empty value to separate methods and skips them if dbm.ndbm uses Berkeley DB.
msg315892 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-29 12:45
New changeset 70af06cdc4e8fbee0b9d7d46bdc193097d4bc71f by Serhiy Storchaka in branch 'master':
bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. (GH-6632)
https://github.com/python/cpython/commit/70af06cdc4e8fbee0b9d7d46bdc193097d4bc71f
msg315896 - (view) Author: miss-islington (miss-islington) Date: 2018-04-29 13:37
New changeset e57d3e3a13d1d4c6a057e47db8361695194bb5ab by Miss Islington (bot) in branch '3.7':
bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. (GH-6632)
https://github.com/python/cpython/commit/e57d3e3a13d1d4c6a057e47db8361695194bb5ab
msg315897 - (view) Author: miss-islington (miss-islington) Date: 2018-04-29 13:47
New changeset a26a297b4ba1b8fe6c97c25af71216935960b343 by Miss Islington (bot) in branch '3.6':
bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. (GH-6632)
https://github.com/python/cpython/commit/a26a297b4ba1b8fe6c97c25af71216935960b343
msg355591 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2019-10-28 20:13
For what it's worth, I'm cherry-picking this back into 3.5 for 3.5.8 final.  I (finally?) got bit by this, and since the patch is literally only changes in the Lib/test directory I consider it safe to merge even after 3.5.8rc2.  (I was in a bit of a hurry, I didn't use the Python cherry picker technology, I just used "git cherry-pick".)
msg355629 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2019-10-29 08:00
New changeset d7b336fe5d54f73c758802df426e06e8a674bd63 by Larry Hastings (Serhiy Storchaka) in branch '3.5':
[3.5] bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. (GH-6632)
https://github.com/python/cpython/commit/d7b336fe5d54f73c758802df426e06e8a674bd63
History
Date User Action Args
2019-10-29 08:00:50larrysetmessages: + msg355629
2019-10-28 20:13:04larrysetnosy: + larry
messages: + msg355591
2018-04-29 14:08:20serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-04-29 13:47:42miss-islingtonsetmessages: + msg315897
2018-04-29 13:37:23miss-islingtonsetnosy: + miss-islington
messages: + msg315896
2018-04-29 12:48:15miss-islingtonsetpull_requests: + pull_request6331
2018-04-29 12:46:28miss-islingtonsetpull_requests: + pull_request6330
2018-04-29 12:45:05serhiy.storchakasetmessages: + msg315892
2018-04-29 11:49:19serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request6328
2018-04-29 11:48:20serhiy.storchakasetmessages: + msg315888
versions: + Python 3.8
2017-09-08 23:15:03bmoylessetmessages: + msg301756
2017-09-08 21:43:06bmoylessetnosy: + bmoyles
messages: + msg301740
2017-07-29 00:14:22terry.reedysetnosy: + terry.reedy
messages: + msg299442
2017-07-25 13:05:24serhiy.storchakasetmessages: + msg299073
2017-07-25 12:22:28vstinnersetmessages: + msg299061
2017-07-25 12:19:18vstinnersetnosy: + vstinner
messages: + msg299058
2017-07-25 12:14:58serhiy.storchakasettitle: test_bisect fails when run directly -> test_dbm fails when run directly
2017-07-25 12:10:17serhiy.storchakacreate