classification
Title: CheckTraceCallbackContent of test_sqlite3 fails on OS X Tiger
Type: Stage: resolved
Components: macOS, Tests Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: berker.peksag, db3l, ned.deily, palaviv, ronaldoussoren, vstinner
Priority: normal Keywords:

Created on 2017-04-21 12:26 by vstinner, last changed 2017-05-02 14:38 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 1238 closed vstinner, 2017-04-21 12:32
Messages (10)
msg292032 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-21 12:26
I suggest to skip the following test on OS X Tiger, since it fails but I'm not interested to fix it and it seems to be an old SQLite bug which was fixed after Tiger was released (I don't see this failure on any other buildbot).

http://buildbot.python.org/all/builders/x86%20Tiger%203.x/builds/569/steps/test/logs/stdio

======================================================================
FAIL: CheckTraceCallbackContent (sqlite3.test.hooks.TraceCallbackTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/sqlite3/test/hooks.py", line 270, in CheckTraceCallbackContent
    self.assertEqual(traced_statements, queries)
AssertionError: Lists differ: ['cre[19 chars]'insert into foo(x) values(1)', 'insert into foo(x) values(1)'] != ['cre[19 chars]'insert into foo(x) values(1)']

First list contains 1 additional elements.
First extra element 2:
'insert into foo(x) values(1)'

+ ['create table foo(x)', 'insert into foo(x) values(1)']
- ['create table foo(x)',
-  'insert into foo(x) values(1)',
-  'insert into foo(x) values(1)']

----------------------------------------------------------------------
msg292033 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-21 12:30
SQLite version:

Re-running test 'test_sqlite' in verbose mode
test_sqlite: testing with version '2.6.0', sqlite_version '3.6.11'
msg292035 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-21 12:33
The test is already skipped on SQLite version older than 3.3.9, no idea why the test fails on SQLite version 3.6.11 on OS X Tiger:

@unittest.skipIf(sqlite.sqlite_version_info < (3, 3, 9),
                 "sqlite3_prepare_v2 is not available") 

I proposed the PR https://github.com/python/cpython/pull/1238
msg292036 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-21 12:37
It's even more strange. The test started to fail since this build, Sun Apr 9 10:10:15 2017:

http://buildbot.python.org/all/builders/x86%20Tiger%203.x/builds/528

test_sqlite: testing with version '2.6.0', sqlite_version '3.6.11'
...
test test_sqlite failed


Sadly, before the test passed and so the SQLite version wasn't logged:

http://buildbot.python.org/all/builders/x86%20Tiger%203.x/builds/527/steps/test/logs/stdio
0:26:24 [227/404/1] test_sqlite passed


@David Bolen: Did you make a change, like an upgrade, on this buildbot slave around Apr 9, 2017?
msg292052 - (view) Author: Aviv Palivoda (palaviv) * Date: 2017-04-21 14:54
> It's even more strange. The test started to fail since this build, Sun Apr 9 10:10:15 2017:

This was merged on Apr 9 (commit 0e6cb2ea624570ed08c354f1ed1f595dab4192d6).

From a quick look in the internet some other people had a similar problem:
http://stackoverflow.com/questions/28973887/how-to-fix-sqlite3-libversion-mismatch

It seems there might be mismatch between sqlite3_libversion() and SQLITE_VERSION. I am not familiar with the buildbots but can you run a test that check that sqlite3_libversion() == SQLITE_VERSION?
msg292055 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-21 15:34
> It seems there might be mismatch between sqlite3_libversion() and SQLITE_VERSION. I am not familiar with the buildbots but can you run a test that check that sqlite3_libversion() == SQLITE_VERSION?

The Python sqlite3 exports sqlite3_libversion() as sqlite3.sqlite_version, "3.6.11" on OS X Tiger.

The doc https://sqlite.org/c3ref/libversion.html says "These interfaces [sqlite3_libversion()] provide the same information as the SQLITE_VERSION, ..."

I don't know how to check, I don't have access to the buildbot.
msg292075 - (view) Author: David Bolen (db3l) Date: 2017-04-21 19:39
No recent changes to the buildbot (I think the last was in September to bump the local python used to run the slave/build installer to 2.7).

The default system sqlite (/usr/lib) is 3.0.8.6, so extremely old, but 3.6.11 is in /usr/local/{lib,include} from early days of setting up the slave, so only marginally less old.  As long as the python build process finds that (which it appears to) it should be consistent.

I configured and built a local 3.x branch version of python on the slave and modified it to dump the version info during module initialization:

> ./python.exe
Python 3.7.0a0 (heads/master:d1ae24e, Apr 21 2017, 15:06:29) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
SQLITE_VERSION: 3.6.11
SQLITE_VERSION_NUMBER: 3006011
sqlite3_libversion: 3.6.11

so it appears to be in line.  This local build still fails the test.

Maybe whatever feature is needed for this test is broken with 3.6.11.

One thing I did notice, though I suspect it's not an issue, but the 3.6.11 this slave is using seems to fall into the gap in the original bpo-9303 _v2 support patch where a _v2 prepare with a plain close (versions >= 3.3.9 and <3.7.14).  Both plain and _v2 close methods have the same signature, but I wonder if there might be an issue mixing and matching?

I don't think anything else has a hard dependency on my /usr/local libraries on the slave at this point (the dmg installer builds its own local copies), so I could try updating that to a more current sqlite3 if we wanted to see if that resolved the issue.
msg292076 - (view) Author: David Bolen (db3l) Date: 2017-04-21 20:07
The test appears to pass against a local test build of sqlite3 3.18.0, so I'll probably just plan on updating the slave to that later tonight.  Unless anyone is interested in figuring out why it fails with 3.6.11 and not 3.18.0 (e.g., does the test have a higher minimum version than it currently declares).
msg292087 - (view) Author: David Bolen (db3l) Date: 2017-04-21 23:49
Ok, this should be resolved.  I opted to be a bit more conservative and only upgraded to sqlite3 3.8.3.1, which is the oldest version still in use for the dmg installer packages (for 2.7).  I restarted the most recent 3.x build and while it's not completely done yet, the sqlite tests have passed.
msg292766 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-02 14:38
David: "Ok, this should be resolved."

Oh, thank you! I confirm that test_sqlite3 pass again!
History
Date User Action Args
2017-05-02 14:38:20vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg292766

stage: resolved
2017-04-26 11:19:46berker.peksagsetnosy: + berker.peksag
2017-04-21 23:49:17db3lsetmessages: + msg292087
2017-04-21 20:07:26db3lsetmessages: + msg292076
2017-04-21 19:39:31db3lsetmessages: + msg292075
2017-04-21 15:34:06vstinnersetmessages: + msg292055
2017-04-21 14:54:28palavivsetnosy: + palaviv
messages: + msg292052
2017-04-21 12:37:20vstinnersetnosy: + db3l
messages: + msg292036
2017-04-21 12:33:50vstinnersetmessages: + msg292035
2017-04-21 12:32:00vstinnersetpull_requests: + pull_request1356
2017-04-21 12:30:20vstinnersetmessages: + msg292033
2017-04-21 12:26:34vstinnercreate