[v2] sqlite3: upgrade 3.48.0 -> 3.50.1

This update includes major change in how it is built.
Instead of autotools, autosetup is used.

Autosetup (https://msteveb.github.io/autosetup/) claims to be
* Replacement for autoconf in many situations
However it also claims NOT to
* Intended to replace all possible uses of autoconf
This means that some autoconf features are not available.

Recipe changes:
* stop inheriting autotools and define B, do_configure and do_install
* depend on zlib unconditionally, autoconf cannot be preconfigured in
  similar way as autotools
* update packageconfig options to match new syntax
* libedit is detected with ncurses linking options (as seen in
  do_configure log)
* backport rpaths fix
* define soname to avoid file-rdeps QA error due to wrong library name
* add hack to force cross-compilation in native case to link against
  zlib in sysroot and thus avoid crashes when sstate-cache from different
  distro is used
* clean B for do_configure as the new Makefiles do not seem to properly
  retrigger build if configuration changes

Kudos to Peter Marko for the initial work on upgrading SQLite

Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
---
 meta/recipes-support/sqlite/sqlite3.inc       | 43 ++++++++++---
 ...rpath-configure-script-flag-to-addre.patch | 60 +++++++++++++++++++
 .../{sqlite3_3.48.0.bb => sqlite3_3.50.1.bb}  |  4 +-
 3 files changed, 98 insertions(+), 9 deletions(-)
 create mode 100644 meta/recipes-support/sqlite/sqlite3/0001-Add-the-disable-rpath-configure-script-flag-to-addre.patch
 rename meta/recipes-support/sqlite/{sqlite3_3.48.0.bb => sqlite3_3.50.1.bb} (53%)

On Fri Jun 27, 2025 at 8:12 PM CEST, Guðni Már Gilbert via lists.openembedded.org wrote:
> This update includes major change in how it is built.
> Instead of autotools, autosetup is used.
>
> Autosetup (https://msteveb.github.io/autosetup/) claims to be
> * Replacement for autoconf in many situations
> However it also claims NOT to
> * Intended to replace all possible uses of autoconf
> This means that some autoconf features are not available.
>
> Recipe changes:
> * stop inheriting autotools and define B, do_configure and do_install
> * depend on zlib unconditionally, autoconf cannot be preconfigured in
>   similar way as autotools
> * update packageconfig options to match new syntax
> * libedit is detected with ncurses linking options (as seen in
>   do_configure log)
> * backport rpaths fix
> * define soname to avoid file-rdeps QA error due to wrong library name
> * add hack to force cross-compilation in native case to link against
>   zlib in sysroot and thus avoid crashes when sstate-cache from different
>   distro is used
> * clean B for do_configure as the new Makefiles do not seem to properly
>   retrigger build if configuration changes
>
> Kudos to Peter Marko for the initial work on upgrading SQLite
>
> Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
> ---
>  meta/recipes-support/sqlite/sqlite3.inc       | 43 ++++++++++---
>  ...rpath-configure-script-flag-to-addre.patch | 60 +++++++++++++++++++
>  .../{sqlite3_3.48.0.bb => sqlite3_3.50.1.bb}  |  4 +-
>  3 files changed, 98 insertions(+), 9 deletions(-)
>  create mode 100644 meta/recipes-support/sqlite/sqlite3/0001-Add-the-disable-rpath-configure-script-flag-to-addre.patch
>  rename meta/recipes-support/sqlite/{sqlite3_3.48.0.bb => sqlite3_3.50.1.bb} (53%)
>
> diff --git a/meta/recipes-support/sqlite/sqlite3.inc b/meta/recipes-support/sqlite/sqlite3.inc
> index 28a33282ae..3251d310fb 100644
> --- a/meta/recipes-support/sqlite/sqlite3.inc
> +++ b/meta/recipes-support/sqlite/sqlite3.inc
> @@ -14,34 +14,37 @@ def sqlite_download_version(d):
>  SQLITE_PV = "${@sqlite_download_version(d)}"
>  
>  S = "${UNPACKDIR}/sqlite-autoconf-${SQLITE_PV}"
> +B = "${WORKDIR}/build"
>  
> -UPSTREAM_CHECK_URI = "http://www.sqlite.org/"
> +UPSTREAM_CHECK_URI = "https://www.sqlite.org/"
>  UPSTREAM_CHECK_REGEX = "releaselog/(?P<pver>(\d+[\.\-_]*)+)\.html"
>  
>  CVE_PRODUCT = "sqlite"
>  
> -inherit autotools pkgconfig siteinfo
> +inherit pkgconfig siteinfo
> +
> +# zlib is autodetected and gets to sysroots as transitive dependency, make this deterministic
> +DEPENDS = "zlib"
>  
>  # enable those which are enabled by default in configure
>  PACKAGECONFIG ?= "fts4 fts5 rtree dyn_ext"
>  PACKAGECONFIG:class-native ?= "fts4 fts5 rtree dyn_ext"
>  
> -PACKAGECONFIG[editline] = "--enable-editline,--disable-editline,libedit"
> -PACKAGECONFIG[readline] = "--enable-readline,--disable-readline,readline ncurses"
> +PACKAGECONFIG[editline] = "--enable-editline --with-readline-header=${includedir}/editline/readline.h,--disable-editline,libedit ncurses"
> +PACKAGECONFIG[readline] = "--enable-readline --with-readline-header=${includedir}/readline/readline.h,--disable-readline,readline ncurses"
>  PACKAGECONFIG[fts3] = "--enable-fts3,--disable-fts3"
>  PACKAGECONFIG[fts4] = "--enable-fts4,--disable-fts4"
>  PACKAGECONFIG[fts5] = "--enable-fts5,--disable-fts5"
>  PACKAGECONFIG[rtree] = "--enable-rtree,--disable-rtree"
>  PACKAGECONFIG[session] = "--enable-session,--disable-session"
> -PACKAGECONFIG[dyn_ext] = "--enable-dynamic-extensions,--disable-dynamic-extensions"
> -PACKAGECONFIG[zlib] = ",,zlib"
> -
> -CACHED_CONFIGUREVARS += "${@bb.utils.contains('PACKAGECONFIG', 'zlib', '', 'ac_cv_search_deflate=no',d)}"
> +PACKAGECONFIG[dyn_ext] = "--enable-load-extension,--disable-load-extension"
>  
>  EXTRA_OECONF = " \
>      --enable-shared \
>      --enable-threadsafe \
> +    --disable-rpath \
>      --disable-static-shell \
> +    --soname=${PV} \
>  "
>  
>  # pread() is in POSIX.1-2001 so any reasonable system must surely support it
> @@ -65,4 +68,28 @@ FILES:lib${BPN}-staticdev = "${libdir}/lib*.a"
>  
>  AUTO_LIBNAME_PKGS = "${MLPREFIX}lib${BPN}"
>  
> +do_configure() {
> +    ${S}/configure \
> +        --build=${BUILD_SYS} \
> +        --host=${TARGET_SYS} \
> +        --prefix=${prefix} \
> +        --bindir=${bindir} \
> +        --libdir=${libdir} \
> +        --includedir=${includedir} \
> +        --mandir=${mandir} \
> +        ${EXTRA_OECONF} \
> +        ${PACKAGECONFIG_CONFARGS}
> +}
> +do_configure[cleandirs] = "${B}"
> +
> +do_install() {
> +    oe_runmake DESTDIR=${D} install
> +
> +    # binaries are stripped during installation when not cross-compiling, take the unstripped ones instead
> +    if [ "${BUILD_SYS}" = "${TARGET_SYS}" ]; then
> +        install -m 0644 ${B}/sqlite3 ${D}${bindir}
> +        install -m 0644 ${B}/libsqlite3.so ${D}${libdir}/libsqlite3.so.${PV}
> +    fi
> +}
> +
>  BBCLASSEXTEND = "native nativesdk"
> diff --git a/meta/recipes-support/sqlite/sqlite3/0001-Add-the-disable-rpath-configure-script-flag-to-addre.patch b/meta/recipes-support/sqlite/sqlite3/0001-Add-the-disable-rpath-configure-script-flag-to-addre.patch
> new file mode 100644
> index 0000000000..9625b3045d
> --- /dev/null
> +++ b/meta/recipes-support/sqlite/sqlite3/0001-Add-the-disable-rpath-configure-script-flag-to-addre.patch
> @@ -0,0 +1,60 @@
> +From 509f5574267c8353a10ff81e96d8393248810b80 Mon Sep 17 00:00:00 2001
> +From: stephan <stephan@noemail.net>
> +Date: Sun, 22 Jun 2025 22:48:11 +0000
> +Subject: [PATCH] Add the --disable-rpath configure script flag to address
> + [forum:13cac3b56516f849 | forum post 13cac3b56516f849].
> +
> +FossilOrigin-Name: a59d9bb25e518f5d79f654615b92f6c50cfb704b5abee0f820912644b89366c5
> +
> +Upstream-Status: Backport [https://github.com/sqlite/sqlite/commit/87c807c6dd4df67328919fa28e89a06839e634fe]
> +Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
> +---
> + autosetup/sqlite-config.tcl | 18 +++++++++---------
> + 1 file changed, 9 insertions(+), 9 deletions(-)
> +
> +diff --git a/autosetup/sqlite-config.tcl b/autosetup/sqlite-config.tcl
> +index 85fe414382..8409dbdd81 100644
> +--- a/autosetup/sqlite-config.tcl
> ++++ b/autosetup/sqlite-config.tcl
> +@@ -334,8 +334,8 @@ proc sqlite-configure {buildMode configScript} {
> +           => {Link the sqlite3 shell app against the DLL instead of embedding sqlite3.c}
> +       }
> +       {canonical autoconf} {
> +-        # A potential TODO without a current use case:
> +-        #rpath=1 => {Disable use of the rpath linker flag}
> ++        rpath=1 => {Disable use of the rpath linker flag}
> ++
> +         # soname: https://sqlite.org/src/forumpost/5a3b44f510df8ded
> +         soname:=legacy
> +           => {SONAME for libsqlite3.so. "none", or not using this flag, sets no
> +@@ -2119,7 +2119,6 @@ proc sqlite-handle-tcl {} {
> + ########################################################################
> + # Handle the --enable/disable-rpath flag.
> + proc sqlite-handle-rpath {} {
> +-  proj-check-rpath
> +   # autosetup/cc-shared.tcl sets the rpath flag definition in
> +   # [get-define SH_LINKRPATH], but it does so on a per-platform basis
> +   # rather than as a compiler check. Though we should do a proper
> +@@ -2128,12 +2127,13 @@ proc sqlite-handle-rpath {} {
> +   # for which sqlite-env-is-unix-on-windows returns a non-empty
> +   # string.
> + 
> +-#  if {[proj-opt-truthy rpath]} {
> +-#    proj-check-rpath
> +-#  } else {
> +-#    msg-result "Disabling use of rpath."
> +-#    define LDFLAGS_RPATH ""
> +-#  }
> ++  # https://sqlite.org/forum/forumpost/13cac3b56516f849
> ++  if {[proj-opt-truthy rpath]} {
> ++    proj-check-rpath
> ++  } else {
> ++    msg-result "Disabling use of rpath."
> ++    define LDFLAGS_RPATH ""
> ++  }
> + }
> + 
> + ########################################################################
> +-- 
> +2.43.0
> +
> diff --git a/meta/recipes-support/sqlite/sqlite3_3.48.0.bb b/meta/recipes-support/sqlite/sqlite3_3.50.1.bb
> similarity index 53%
> rename from meta/recipes-support/sqlite/sqlite3_3.48.0.bb
> rename to meta/recipes-support/sqlite/sqlite3_3.50.1.bb
> index bd2ac6614d..bf2e883082 100644
> --- a/meta/recipes-support/sqlite/sqlite3_3.48.0.bb
> +++ b/meta/recipes-support/sqlite/sqlite3_3.50.1.bb
> @@ -4,5 +4,7 @@ LICENSE = "PD"
>  LIC_FILES_CHKSUM = "file://sqlite3.h;endline=11;md5=786d3dc581eff03f4fd9e4a77ed00c66"
>  
>  SRC_URI = "http://www.sqlite.org/2025/sqlite-autoconf-${SQLITE_PV}.tar.gz"
> -SRC_URI[sha256sum] = "ac992f7fca3989de7ed1fe99c16363f848794c8c32a158dafd4eb927a2e02fd5"
> +SRC_URI[sha256sum] = "00a65114d697cfaa8fe0630281d76fd1b77afcd95cd5e40ec6a02cbbadbfea71"
> +
> +SRC_URI += "file://0001-Add-the-disable-rpath-configure-script-flag-to-addre.patch"
>  

Hi,

This patch introduces errors such as the following on the Autobuilder:

Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/beaglebone/build/bitbake/lib/bb/server/process.py", line 450, in idle_thread_internal
    retval = function(self, data, False)
  File "/srv/pokybuild/yocto-worker/beaglebone/build/bitbake/lib/bb/cooker.py", line 1493, in buildTargetsIdle
    retval = rq.execute_runqueue()
  File "/srv/pokybuild/yocto-worker/beaglebone/build/bitbake/lib/bb/runqueue.py", line 1659, in execute_runqueue
    return self._execute_runqueue()
  File "/srv/pokybuild/yocto-worker/beaglebone/build/bitbake/lib/bb/runqueue.py", line 1625, in _execute_runqueue
    retval = self.rqexe.execute()
  File "/srv/pokybuild/yocto-worker/beaglebone/build/bitbake/lib/bb/runqueue.py", line 2315, in execute
    self.rq.start_fakeworker(self, mc)
  File "/srv/pokybuild/yocto-worker/beaglebone/build/bitbake/lib/bb/runqueue.py", line 1435, in start_fakeworker
    self.fakeworker[mc] = self._start_worker(mc, True, rqexec)
  File "/srv/pokybuild/yocto-worker/beaglebone/build/bitbake/lib/bb/runqueue.py", line 1404, in _start_worker
    RunQueue.send_pickled_data(worker, workerdata, "workerdata")
  File "/srv/pokybuild/yocto-worker/beaglebone/build/bitbake/lib/bb/runqueue.py", line 1363, in send_pickled_data
    worker.stdin.write(msg)
BrokenPipeError: [Errno 32] Broken pipe

I didn't have the time to reproduce the issue locally, but bisecting shows that
this patch triggers such errors pretty on almost all builders. See this build
for example:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/1957

Antonin

Thank for taking the time to test this Antonin. I saw this error locally rarely, and usually deleting the entire build folder resolved the problem.

I am not sure how to fix this issue, if anybody has any ideas then I would gladly listen/appreciate it. I'm not very familiar with the underlying Python code that is detecting a broken pipe. I wonder if this is some sort of race condition?

To add, I could send another patch to update from v3.50.1 to v3.50.2, but I doubt that the problem is with the SQLite source files. It has to be something related to the Bitbake recipe right?

On Wed, 2025-07-02 at 01:49 -0700, Guðni Már Gilbert via
lists.openembedded.org wrote:
> To add, I could send another patch to update from v3.50.1 to v3.50.2,
> but I doubt that the problem is with the SQLite source files. It has
> to be something related to the Bitbake recipe right?

The error is occurring in start_fakeworker() which means it is trying
to call "pseudo bitbake-worker" and that is failing for some reason.

pseudo is a heavy users of sqlite-native so I'd suspect the problem is
somewhere in pseudo's use of sqlite.

Cheers,

Richard