From patchwork Thu Jul 30 23:16:22 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 93964 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FBE5C5516D for ; Thu, 30 Jul 2026 23:16:57 +0000 (UTC) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.165.1785453413983907597 for ; Thu, 30 Jul 2026 16:16:54 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=AganuiVs; spf=pass (domain: smile.fr, ip: 209.85.128.48, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-49800c6a846so1697825e9.3 for ; Thu, 30 Jul 2026 16:16:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1785453412; x=1786058212; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=A+nEnqi5z4PbqdCciUaUJ6Gv4lH4n6mk+yjWjAFyHV0=; b=AganuiVsRonRrNMGJ2462Rc+6XDTQYfZr06kIvCvAVWdYi2Mj0zEpz1PGmeV6W1F7s JMAp4fx/tTGzlbq2cJ+O6UM+G+UCcsCcVb6kpg3G5XxB+mIVh37W/MkFypQ9Uu/8DepR ZPe3en/ZXYqMej54+K2CJR9c8quBfar3muwJE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785453412; x=1786058212; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=A+nEnqi5z4PbqdCciUaUJ6Gv4lH4n6mk+yjWjAFyHV0=; b=ltBy7HBahHHc0fthbValjoStImfKNMmcElmW4W+lZhaaBQlg8FzWN5Q7nw9cni2P4q BVPgvrlgURvWgwjKOwmdU6j110tOSgDjB1gQ+nIWHA5gkRESFJo2zbnCnIJZlH/Hsxsa Lr3QECNI4mmumvt0QBM2HA/XazTnowqpT6+uuopAQJsSyzW8MFFMSoxIqSQsL6mX9Ab3 zjZHP9Asp+rO5sRQE0aVVJdK3j8IERpuqYao04zmhsQ1AozRHFF/Xp5aQuNk5aI08UkX 3FudJEVaKfCrwZ6Kmbxwhk0h7CTDMj6+I0d1bQJkJTLT3f81TxTZlt9C+AfGqk8TTOZQ mcJw== X-Gm-Message-State: AOJu0YysK2Tk0uHjngZkafgT7Q07s+1X4xqBsvRXtxBeooaanozAarB3 JWTyLhYnZFt90HC/LDZgS0yEL8lqwon9jOu/EEKKQNVWmlMVMxdRzE02oXqDTQ4fQhWne1SMbBE gTemVP00= X-Gm-Gg: AR+sD10n3BoWUZClI93lmSb6aCZ0/lmPSnJ+1ql7X6K3V5UQJHnTZniNRcuze4Ry2Lb Y20jX6AOtO+y38CwQ1sxwmkQwDZJU73/OpUljmIKJcTxyHs4ku73G/NxumW38vqsX02l0IZP+DI VvG2r1VDWwV5HASGmnI7TenP00YjmuR5MRcvyi6sFakaHqe3Kso3cJZ3H3mecUZdddn6YFpRc1h Kujc6D0Q8N3G+og25i9MPCmtveIuqIsgWCFb/4HKy6NXYbMPJLEK6zYkO8ucQmcS5pVz4sGmCx2 ICXBy9fKxadybd2AQsaq7K5nrmgx4JDm7ms8fdagzyfo+qAtcVKqzZeOYpc/tbaq2wwZyg38Siw vXs1ba3DHkw8m7z4mAURlZ6DLzHzML9me8MjHWokFNuKGN1I1+M+qT7BczrASamPJJZ07/7PAOq R7Z4GACEC/LYWSBa8B9258fUA10OZ2M2wddiVNTnnAHAlhXfwzY2DOERkJbsZBTI4Ae/EQFAiWc NXuwyFIAogC+j173muGmfafGAnT7HdY6W7UHnDCzMDpnrsbjrupTt0Wg9UJWN6K X-Received: by 2002:a05:600c:6305:b0:496:bba5:33a8 with SMTP id 5b1f17b1804b1-49800eb9dd7mr67734065e9.33.1785453412191; Thu, 30 Jul 2026 16:16:52 -0700 (PDT) Received: from FRSMI25-LASER.home (2a01cb001331aa00a2e4fb7b0d887544.ipv6.abo.wanadoo.fr. [2a01:cb00:1331:aa00:a2e4:fb7b:d88:7544]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-498062fb5a1sm6423525e9.3.2026.07.30.16.16.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jul 2026 16:16:51 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: Richard Purdie Subject: [OE-core][kirkstone 1/2] cve_check: Use a local copy of the database during builds Date: Fri, 31 Jul 2026 01:16:22 +0200 Message-ID: X-Mailer: git-send-email 2.47.3 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 30 Jul 2026 23:16:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/242383 From: Richard Purdie Rtaher than trying to use a sqlite database over NFS from DL_DIR, work from a local copy in STAGING DIR after fetching. Signed-off-by: Richard Purdie (cherry picked from commit 03596904392d257572a905a182b92c780d636744) [YC: Fixing [YOCTO #15660] ] Signed-off-by: Yoann Congal --- meta/classes/cve-check.bbclass | 7 ++++--- .../meta/cve-update-nvd2-native.bb | 18 +++++++++++++----- 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass index 5e6bae17570..19d7ef5d852 100644 --- a/meta/classes/cve-check.bbclass +++ b/meta/classes/cve-check.bbclass @@ -25,8 +25,9 @@ CVE_PRODUCT ??= "${BPN}" CVE_VERSION ??= "${PV}" -CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK" -CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_2.db" +CVE_CHECK_DB_FILENAME ?= "nvdcve_2.db" +CVE_CHECK_DB_DIR ?= "${STAGING_DIR}/CVE_CHECK" +CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/${CVE_CHECK_DB_FILENAME}" CVE_CHECK_DB_FILE_LOCK ?= "${CVE_CHECK_DB_FILE}.lock" CVE_CHECK_LOG ?= "${T}/cve.log" @@ -156,7 +157,7 @@ python do_cve_check () { } addtask cve_check before do_build -do_cve_check[depends] = "cve-update-nvd2-native:do_fetch" +do_cve_check[depends] = "cve-update-nvd2-native:do_unpack" do_cve_check[nostamp] = "1" python cve_check_cleanup () { diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 1a3eeba6d0c..4f96883beba 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -8,7 +8,6 @@ INHIBIT_DEFAULT_DEPS = "1" inherit native -deltask do_unpack deltask do_patch deltask do_configure deltask do_compile @@ -35,7 +34,9 @@ CVE_DB_INCR_UPDATE_AGE_THRES ?= "10368000" # Number of attempts for each http query to nvd server before giving up CVE_DB_UPDATE_ATTEMPTS ?= "5" -CVE_DB_TEMP_FILE ?= "${CVE_CHECK_DB_DIR}/temp_nvdcve_2.db" +CVE_CHECK_DB_DLDIR_FILE ?= "${DL_DIR}/CVE_CHECK/${CVE_CHECK_DB_FILENAME}" +CVE_CHECK_DB_DLDIR_LOCK ?= "${CVE_CHECK_DB_DLDIR_FILE}.lock" +CVE_CHECK_DB_TEMP_FILE ?= "${CVE_CHECK_DB_FILE}.tmp" python () { if not bb.data.inherits_class("cve-check", d): @@ -52,9 +53,9 @@ python do_fetch() { bb.utils.export_proxies(d) - db_file = d.getVar("CVE_CHECK_DB_FILE") + db_file = d.getVar("CVE_CHECK_DB_DLDIR_FILE") db_dir = os.path.dirname(db_file) - db_tmp_file = d.getVar("CVE_DB_TEMP_FILE") + db_tmp_file = d.getVar("CVE_CHECK_DB_TEMP_FILE") cleanup_db_download(db_file, db_tmp_file) # By default let's update the whole database (since time 0) @@ -77,6 +78,7 @@ python do_fetch() { pass bb.utils.mkdirhier(db_dir) + bb.utils.mkdirhier(os.path.dirname(db_tmp_file)) if os.path.exists(db_file): shutil.copy2(db_file, db_tmp_file) @@ -89,10 +91,16 @@ python do_fetch() { os.remove(db_tmp_file) } -do_fetch[lockfiles] += "${CVE_CHECK_DB_FILE_LOCK}" +do_fetch[lockfiles] += "${CVE_CHECK_DB_DLDIR_LOCK}" do_fetch[file-checksums] = "" do_fetch[vardeps] = "" +python do_unpack() { + import shutil + shutil.copyfile(d.getVar("CVE_CHECK_DB_DLDIR_FILE"), d.getVar("CVE_CHECK_DB_FILE")) +} +do_unpack[lockfiles] += "${CVE_CHECK_DB_DLDIR_LOCK} ${CVE_CHECK_DB_FILE_LOCK}" + def cleanup_db_download(db_file, db_tmp_file): """ Cleanup the download space from possible failed downloads From patchwork Thu Jul 30 23:16:23 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 93965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00467C55167 for ; Thu, 30 Jul 2026 23:16:56 +0000 (UTC) Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.147.1785453414372002400 for ; Thu, 30 Jul 2026 16:16:54 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=Ox/I7W9r; spf=pass (domain: smile.fr, ip: 209.85.128.54, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-495590dde14so2395615e9.0 for ; Thu, 30 Jul 2026 16:16:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1785453412; x=1786058212; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=M6pdFCAF/EpiwrcYrzDSPWIvl5ipxZ7tMrDc5lxWDwo=; b=Ox/I7W9r1/wQqIl1Jg8QMj7aBiso4Envkx7gYAlUg5Qj0fCU+w4Pp1tzDNgRpqm8co z/VmPpHwa983UOBdG5feMpKCteLQGsbW9HF9u18QuwvFt4OE7MOPq0p8qo1nb9KM4oga W3bpC/YV5Dku3bOqGBinmjqKUUvJqYPyslj2Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785453412; x=1786058212; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=M6pdFCAF/EpiwrcYrzDSPWIvl5ipxZ7tMrDc5lxWDwo=; b=Il9HI+6cSL2vtkmGyznBlDmYlwM1hR0VZVL2FmGgQzbSxUETKa1rBAR1aa26+dXQFV COC6YiuyY0I5MX/7NvV+OcZCZSpWr4mdfYuMVC7Ixi/ysLOU0w9bxF/KY7cp257VY6KG jYFuTYOJuR/0sMcalZYGRXTkgEcqS/LxLzgjTxvZhR1TIscO6B/V0HbHr6HsYlTFv8Aa fUnobS3GyIDEPoWu8tqhs3tdr6qvD9ASCrOho3Sbqt6n6IFn3qEKGOpAnI/Ay0kNJGqq zMncY5wX4y7YGrulz/Wd2KDJQ/Njmqh5bTqbzwNRrmvPqiw5Ev6YJmEEVMyB2B3QJGbq rYBw== X-Gm-Message-State: AOJu0Yy2fLrBAjDhjyaf9BhBPJm+KnHPVW/dE6BJ7FtH8OvTxyT4x1s/ N1WntQit5iGeMNA7m5R3IMZlTkgQjkXWTSpMzOajwq2yd0IW9eobQ2UDGPZyTKfsA4xaXdY2XP2 +39kj3Js= X-Gm-Gg: AR+sD13HwrXi1GZAXfTcJ4PcZr23FuMRVjnRz7ciBB2Nh6apb9mR7ppTh0hMLH2bLha jLRKRoInUnYyM+pATbdq1PaR0CGbQLq1rHOktGZ9DzI7QSihFtCiRrr7ye1bV6kFF5Lp43C4FKr noYGjDhrxySTRau/QvdUd25c98eBY7MGF1VlIywqqsPz2OixaKYw7imor3K7ofKFIMzbF6nWEFz muZzq7Hw47zs37Qciwjs7DQrsFPCUTV8cZ0tRyQ5dWSeQtGxmuLYq+fKjG9LFj8pQu2NSq0D4Eb RZ6P0od7tgPPRw61wUOppqTFHiQDuEdXyrlgdoa0hvkTzZknmAirZh9FaTQuMcJfP7ZWdO4n522 ocRAwE9omPvbe9kbKWCPwtiGgclthjGwYp2JhKrmz7wC/wazl6jziSoJuLSs4Cxa9oBQ1er9k7I AtNB/hSR0X1FuQObcN/qKYma/Xbo8SO2M76ecGmdR4fPz14CfIf92+k/GkUdqETbYawGvabDs7d 4NDzw/44KCh7/Y0J8VLlK6k+MbfIXxeH/2NgkNvJN5Qml2Gxx3ekbl2Dl7kfnOh8c6eHwPoyS8= X-Received: by 2002:a05:600c:3b8e:b0:490:44eb:c1ea with SMTP id 5b1f17b1804b1-49804c532damr33875035e9.24.1785453412621; Thu, 30 Jul 2026 16:16:52 -0700 (PDT) Received: from FRSMI25-LASER.home (2a01cb001331aa00a2e4fb7b0d887544.ipv6.abo.wanadoo.fr. [2a01:cb00:1331:aa00:a2e4:fb7b:d88:7544]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-498062fb5a1sm6423525e9.3.2026.07.30.16.16.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jul 2026 16:16:52 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: Richard Purdie Subject: [OE-core][kirkstone 2/2] cve-update: Avoid NFS caching issues Date: Fri, 31 Jul 2026 01:16:23 +0200 Message-ID: <1e668895f80725cb4102d000c879d57464757cc1.1785450470.git.yoann.congal@smile.fr> X-Mailer: git-send-email 2.47.3 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 30 Jul 2026 23:16:56 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/242384 From: Paul Barker When moving the updated CVE database file to the downloads directory, ensure that it has a different inode number to the previous version of this file. We have seen "sqlite3.DatabaseError: database disk image is malformed" exceptions on our autobuilder when trying to read the CVE database in do_cve_check tasks. The context here is that the downloads directory (where the updated database file is copied to) is shared between workers as an NFS mount. Different autobuilder workers were seeing different checksums for the database file, which indicates that a mix of both new and stale data was being read. Forcing each new version of the database file to have a different inode number will prevent stale data from being read from local caches. This should fix [YOCTO #16086]. Signed-off-by: Paul Barker Signed-off-by: Richard Purdie (cherry picked from commit f63622bbec1cfaca6d0b3e05e11466e4c10fa86e) [YC: Fixing [YOCTO #15660] ] Signed-off-by: Yoann Congal --- meta/recipes-core/meta/cve-update-db-native.bb | 9 +++++++-- meta/recipes-core/meta/cve-update-nvd2-native.bb | 9 +++++++-- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index efc32470d31..4255b125bd7 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -63,8 +63,13 @@ python do_fetch() { shutil.copy2(db_file, db_tmp_file) if update_db_file(db_tmp_file, d) == True: - # Update downloaded correctly, can swap files - shutil.move(db_tmp_file, db_file) + # Update downloaded correctly, we can swap files. To avoid potential + # NFS caching issues, ensure that the destination file has a new inode + # number. We do this in two steps as the downloads directory may be on + # a different filesystem to tmpdir we're working in. + new_file = "%s.new" % (db_file) + shutil.move(db_tmp_file, new_file) + os.rename(new_file, db_file) else: # Update failed, do not modify the database bb.note("CVE database update failed") diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 4f96883beba..e4628fca027 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -83,8 +83,13 @@ python do_fetch() { shutil.copy2(db_file, db_tmp_file) if update_db_file(db_tmp_file, d, database_time) == True: - # Update downloaded correctly, can swap files - shutil.move(db_tmp_file, db_file) + # Update downloaded correctly, we can swap files. To avoid potential + # NFS caching issues, ensure that the destination file has a new inode + # number. We do this in two steps as the downloads directory may be on + # a different filesystem to tmpdir we're working in. + new_file = "%s.new" % (db_file) + shutil.move(db_tmp_file, new_file) + os.rename(new_file, db_file) else: # Update failed, do not modify the database bb.warn("CVE database update failed")