Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect SQLite in configure.ac #89932

Closed
erlend-aasland opened this issue Nov 9, 2021 · 11 comments
Closed

Detect SQLite in configure.ac #89932

erlend-aasland opened this issue Nov 9, 2021 · 11 comments
Labels
3.11 only security fixes build The build process and cross-build topic-sqlite3 type-feature A feature request or enhancement

Comments

@erlend-aasland
Copy link
Contributor

BPO 45774
Nosy @tiran, @erlend-aasland
PRs
  • bpo-45774: Autoconfiscate SQLite detection (GH-29507) #29507
  • bpo-45774: Fix SQLite load extension autodetection (GH-29659) #29659
  • bpo-45774: Harden SQLite detection #30016
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2022-04-05.13:56:26.148>
    created_at = <Date 2021-11-09.22:26:46.277>
    labels = ['type-feature', 'build', '3.11']
    title = 'Detect SQLite in configure.ac'
    updated_at = <Date 2022-04-05.13:56:26.148>
    user = 'https://github.com/erlend-aasland'

    bugs.python.org fields:

    activity = <Date 2022-04-05.13:56:26.148>
    actor = 'erlendaasland'
    assignee = 'none'
    closed = True
    closed_date = <Date 2022-04-05.13:56:26.148>
    closer = 'erlendaasland'
    components = ['Build']
    creation = <Date 2021-11-09.22:26:46.277>
    creator = 'erlendaasland'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 45774
    keywords = ['patch']
    message_count = 11.0
    messages = ['406059', '406061', '406066', '406076', '406584', '406585', '406652', '406653', '406655', '408161', '416775']
    nosy_count = 2.0
    nosy_names = ['christian.heimes', 'erlendaasland']
    pr_nums = ['29507', '29659', '30016']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue45774'
    versions = ['Python 3.11']

    @erlend-aasland
    Copy link
    Contributor Author

    "Autoconfiscate" SQLite detection. Plan:

    • Detect header/library using AC_CHECK_HEADERS/AC_CHECK_LIB
    • Check required version using AC_COMPILE_IFELSE
    • Use AC_CHECK_LIB to check if sqlite3_load_extension is present, and if the result harmonises with --enable-loadable-sqlite-extensions; bail if not

    Nice side effect: configure will bail if --enable-loadable-sqlite-extensions is used and the target SQLite library was compiled with SQLITE_OMIT_LOAD_EXTENSION, even if we are not on macOS.

    @erlend-aasland erlend-aasland added 3.11 only security fixes build The build process and cross-build labels Nov 9, 2021
    @tiran
    Copy link
    Member

    tiran commented Nov 9, 2021

    Your solution looks very similar to my WIP patch:

    AC_CHECK_HEADERS([sqlite3.h],
    [AC_SEARCH_LIBS([sqlite3_version], [sqlite3])]
    )

    AS_VAR_IF([ac_cv_search_sqlite3_version], [no], [], [
      # found a working sqlite3.h and sqlite3 library
      # ac_cv_search_sqlite3_version != no
      LIBS="$LIBS $ac_cv_search_sqlite3_version"
      
      AC_CACHE_CHECK([for sqlite3 version >= 3.7.15], [ac_cv_lib_sqlite3_supported], [
        AC_LINK_IFELSE([
          AC_LANG_PROGRAM([
            #include <sqlite3.h>
            #if SQLITE_VERSION_NUMBER < 3007015
              #error "sqlite version is too old"
            #endif], []
          )
        ], [ac_cv_lib_sqlite3_supported=yes], [ac_cv_lib_sqlite3_supported=no])
      ])
      
      AC_CACHE_CHECK([for sqlite3_enable_load_extension], [ac_cv_lib_sqlite3_sqlite3_enable_load_extension], [
        AC_LINK_IFELSE(
          [AC_LANG_PROGRAM([#include <sqlite3.h>], [void *x = sqlite3_enable_load_extension])],
          [ac_cv_lib_sqlite3_sqlite3_enable_load_extension=yes],
          [ac_cv_lib_sqlite3_sqlite3_enable_load_extension=no])
      ])
    ])

    LIBS=$LIBS_SAVE

    AS_VAR_IF([ac_cv_lib_sqlite3_supported], [yes],
    [AC_DEFINE([HAVE_LIBSQLITE3], [1], [Define to 1 if you have the `sqlite3' library (-lsqlite3).])]
    )

    AS_VAR_IF([ac_cv_lib_sqlite3_sqlite3_enable_load_extension], [yes],
    [AC_DEFINE([HAVE_SQLITE3_ENABLE_LOAD_EXTENSION], [1],
    [Define to 1 if `sqlite3' library supports sqlite3_enable_load_extension.])]
    )

    @erlend-aasland
    Copy link
    Contributor Author

    Nice. I think I'll steal some of your ideas tomorrow :)

    @tiran
    Copy link
    Member

    tiran commented Nov 10, 2021

    I saw your message concerning sqlite3 on the FreeBSD buildbot. It's hard to tell why configure on FreeBSD doesn't find sqlite3.h without access to config.log or a shell. Maybe sqlite3 is installed in /usr/local ? configure does not use /usr/local/include and /usr/local/lib by default. setup.py extends the search paths for headers and libraries by non-standard locations.

    A "config.site" file should do the trick:

    $ mkdir -p /usr/local/etc
    $ cat > /usr/local/etc/config.site << EOF
    test -z "$CPPFLAGS" && CPPFLAGS="-I$/usr/local/include"
    test -z "$LDFLAGS" && LDFLAGS="-L/usr/local/lib"
    EOF

    @tiran
    Copy link
    Member

    tiran commented Nov 19, 2021

    New changeset 29e5874 by Erlend Egeberg Aasland in branch 'main':
    bpo-45774: Autoconfiscate SQLite detection (GH-29507)
    29e5874

    @tiran
    Copy link
    Member

    tiran commented Nov 19, 2021

    Awesome work! Thanks to your tireless effort we know have a blue print how to use pkg-config for other librariess.

    @tiran tiran added type-feature A feature request or enhancement labels Nov 19, 2021
    @erlend-aasland
    Copy link
    Contributor Author

    Needs amendment before closing. See #73845.

    @erlend-aasland
    Copy link
    Contributor Author

    Awesome work! Thanks to your tireless effort we know have a blue print how to use pkg-config for other librariess.

    Thank you for reviewing, and thank you for paving the way with all these build system improvements :)

    @tiran
    Copy link
    Member

    tiran commented Nov 20, 2021

    New changeset 6d430ef by Erlend Egeberg Aasland in branch 'main':
    bpo-45774: Fix SQLite load extension autodetection (GH-29659)
    6d430ef

    @erlend-aasland
    Copy link
    Contributor Author

    Reopening: the current detection is a little bit weak; many SQLite API's may be disabled at SQLite compile time using SQLITE_OMIT_* defines. We must make sure we've got all vital functions in place before marking the sqlite3 module as present and usable.

    The following API's can be omitted using compile time defines:

    • sqlite3_column_decltype (SQLITE_OMIT_DECLTYPE)
    • sqlite3_complete (SQLITE_OMIT_COMPLETE)
    • sqlite3_enable_shared_cache (SQLITE_OMIT_SHARED_CACHE)
    • sqlite3_progress_handler (SQLITE_OMIT_PROGRESS_CALLBACK)
    • sqlite3_set_authorizer (SQLITE_OMIT_AUTHORIZATION)
    • sqlite3_trace_v2 (SQLITE_OMIT_TRACE)
    • sqlite3_trace (SQLITE_OMIT_TRACE, SQLITE_OMIT_DEPRECATED)

    The following API's _may_ be disabled in the future using SQLITE_OMIT_FLOATING_POINT:

    • sqlite3_bind_double
    • sqlite3_column_double
    • sqlite3_result_double
    • sqlite3_value_double

    @tiran
    Copy link
    Member

    tiran commented Apr 5, 2022

    New changeset f1606a5 by Erlend Egeberg Aasland in branch 'main':
    bpo-45774: Harden SQLite detection (GH-30016)
    f1606a5

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes build The build process and cross-build topic-sqlite3 type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants