From patchwork Sun Aug 24 14:57:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Marko X-Patchwork-Id: 69086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C14E1CA0EEB for ; Sun, 24 Aug 2025 14:58:41 +0000 (UTC) Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net [185.136.64.225]) by mx.groups.io with SMTP id smtpd.web11.18348.1756047516204537641 for ; Sun, 24 Aug 2025 07:58:36 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm2 header.b=G8XK9x5A; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.225, mailfrom: fm-256628-2025082414583319a10df9dc87d0e410-n7odl2@rts-flowmailer.siemens.com) Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 2025082414583319a10df9dc87d0e410 for ; Sun, 24 Aug 2025 16:58:33 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc; bh=zjt4tBiqSWchByK9h8R+WdSGf4DlnRvV/IxvBsT5+Qg=; b=G8XK9x5AvyUuwREHnHbsXg1eh2/HS9EU9xdBfAdUIxzJ3gEx50X4VTR4IAwcnxOOr7MkfA yCYQQWdmfYJlPwnrvx6u3lTlitbh/h3fvxljIZBC1UdzULMdivGWqo4Sc3ZeVyjwlowy1jI9 Vc1BtdjWdIAF54KBeRMwuCQ8246GirthU3zpbbQ46twiAVldfkJ66/txum5AEzmd8TbSWSeg ZnzLG+E/CdyAzfuj0z3BcO9M4HJqOwYlMk+3A2h6SClXUn9UD7chgYACTmqwC12VPxyBs89H oXPG0cedSqTDTUbJ2iKmlhBxlgGkO5OP6i5s8nFWh4Es2zcPFXm53H7Q==; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Peter Marko Subject: [OE-core][PATCH 1/7] cve-update-db-native: Use a local copy of the database during builds Date: Sun, 24 Aug 2025 16:57:39 +0200 Message-Id: <20250824145745.1099373-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 24 Aug 2025 14:58:41 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/222381 From: Peter Marko OE-Core rev: 03596904392d257572a905a182b92c780d636744 This seems to be misimplemented when re-adding update from nvd1 feed. Use file in temporary directory instead of downloads directory for update process. Signed-off-by: Peter Marko --- meta/recipes-core/meta/cve-update-db-native.bb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index dff5cd1c61..70bef2179f 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -24,8 +24,7 @@ CVE_SOCKET_TIMEOUT ?= "60" CVE_CHECK_DB_DLDIR_FILE ?= "${DL_DIR}/CVE_CHECK2/${CVE_CHECK_DB_FILENAME}" CVE_CHECK_DB_DLDIR_LOCK ?= "${CVE_CHECK_DB_DLDIR_FILE}.lock" - -CVE_DB_TEMP_FILE ?= "${CVE_CHECK_DB_DLDIR_FILE}.tmp" +CVE_CHECK_DB_TEMP_FILE ?= "${CVE_CHECK_DB_FILE}.tmp" python () { if not bb.data.inherits_class("cve-check", d): @@ -44,7 +43,7 @@ python do_fetch() { db_file = d.getVar("CVE_CHECK_DB_DLDIR_FILE") db_dir = os.path.dirname(db_file) - db_tmp_file = d.getVar("CVE_DB_TEMP_FILE") + db_tmp_file = d.getVar("CVE_CHECK_DB_TEMP_FILE") cleanup_db_download(db_file, db_tmp_file) @@ -64,6 +63,7 @@ python do_fetch() { pass bb.utils.mkdirhier(db_dir) + bb.utils.mkdirhier(os.path.dirname(db_tmp_file)) if os.path.exists(db_file): shutil.copy2(db_file, db_tmp_file) From patchwork Sun Aug 24 14:57:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Marko X-Patchwork-Id: 69087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D80BCA0EEB for ; Sun, 24 Aug 2025 14:59:21 +0000 (UTC) Received: from mta-65-225.siemens.flowmailer.net (mta-65-225.siemens.flowmailer.net [185.136.65.225]) by mx.groups.io with SMTP id smtpd.web11.18367.1756047557387973094 for ; Sun, 24 Aug 2025 07:59:18 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm2 header.b=G7yvLzYV; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.225, mailfrom: fm-256628-20250824145914ce434e94e208a991c4-wqw2va@rts-flowmailer.siemens.com) Received: by mta-65-225.siemens.flowmailer.net with ESMTPSA id 20250824145914ce434e94e208a991c4 for ; Sun, 24 Aug 2025 16:59:14 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=ynRdbeCpP5pQgq/KUqLdc38ZgAA1027vYmdHjxK8/eE=; b=G7yvLzYVu6eyPnOLbOd9rZX3m+JLq4PWFM4OyaJTcF2cPQ6qQ3TWewe0qgYqDxx+BVYT5K tAfOS8CWmx+hJ6CA3sBWLJY99eH3RphGD8xq8qFV8SR7zSWOvgTnSCnu67l9JNBgB2favwfx lzkCou2nZK78fBKMqRzRPplolsQ+GcZarV/dSIFjuaKIn4xinL8sML1Au6nAYuK22LT5876z 5H0uePV8jaxg5ryWsUCzPvYy5lHldxgSQxVoP/7wEFDOaoSd8ImIdvOBYrqhdTxD+tow/eBK jZjERgaq9dhJ9j3ok7+QVXKKbXgByG0OlsZ/Ln+Sw1hLHw2zsWHQO3qw==; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Peter Marko Subject: [OE-core][PATCH 2/7] cve-update-db-native: Handle BB_NO_NETWORK and missing db Date: Sun, 24 Aug 2025 16:57:40 +0200 Message-Id: <20250824145745.1099373-2-peter.marko@siemens.com> In-Reply-To: <20250824145745.1099373-1-peter.marko@siemens.com> References: <20250824145745.1099373-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 24 Aug 2025 14:59:21 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/222382 From: Peter Marko OE-Core rev: 337c0806d2784d74bee8d6420fb8b4d48795d5fa This commit was not applied on nvd1/fkie fetcher. Signed-off-by: Peter Marko --- meta/recipes-core/meta/cve-update-db-native.bb | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index 70bef2179f..fe7b8a017f 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -54,6 +54,8 @@ python do_fetch() { update_interval = int(d.getVar("CVE_DB_UPDATE_INTERVAL")) if update_interval < 0: bb.note("CVE database update skipped") + if not os.path.exists(db_file): + bb.error("CVE database %s not present, database fetch/update skipped" % db_file) return if time.time() - os.path.getmtime(db_file) < update_interval: bb.debug(2, "Recently updated, skipping") @@ -62,6 +64,9 @@ python do_fetch() { except OSError: pass + if bb.utils.to_boolean(d.getVar("BB_NO_NETWORK")): + bb.error("BB_NO_NETWORK attempted to disable fetch, this recipe uses CVE_DB_UPDATE_INTERVAL to control download, set to '-1' to disable fetch or update") + bb.utils.mkdirhier(db_dir) bb.utils.mkdirhier(os.path.dirname(db_tmp_file)) if os.path.exists(db_file): From patchwork Sun Aug 24 14:57:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Marko X-Patchwork-Id: 69088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A824ECA0EFC for ; Sun, 24 Aug 2025 14:59:31 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.web11.18370.1756047565054559532 for ; Sun, 24 Aug 2025 07:59:25 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm2 header.b=Xgf3sqtF; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-256628-202508241459231b2e5badb0de42efcd-wamvfm@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 202508241459231b2e5badb0de42efcd for ; Sun, 24 Aug 2025 16:59:23 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=iGd2MqXYKnc/G8apeAT8LxzsXOi6oHDgQkJ6+yAkzmk=; b=Xgf3sqtFFGeAUTg+K3ujK80Uo+XUGcZvpeOy16vj/OA+6u6zbdG4Y4j/eJYNBOWYzffkcm W5MARVqNClw5m0LJntj7OJJ/KkPIWNTHZNa7tAhCJLmRSPAIqOusQFtQ5qWqduORL4o4R342 37BveTC+iP4ocqWRbcbI4Ug4aANRdxAejt1fuGajeurpxTR4oqvtJVoONgmYtB/PALgJDhH5 Wh2wUl1ysCuLZJgViHEKQd3Mx5qpl2hSv/0fZb5QqzUx9TmqFqdDfhKsKhfN7tcQNpVLXeKn +LCHxXy1DsnCuG9fQLuiBKFy/rjHF4VxkZcX6uepEy4IlwucD05ZdS6Q==; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Peter Marko Subject: [OE-core][PATCH 3/7] cve-update-db-native: log a little more Date: Sun, 24 Aug 2025 16:57:41 +0200 Message-Id: <20250824145745.1099373-3-peter.marko@siemens.com> In-Reply-To: <20250824145745.1099373-1-peter.marko@siemens.com> References: <20250824145745.1099373-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 24 Aug 2025 14:59:31 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/222383 From: Peter Marko OE-Core rev: b64a869b9c5e1d504f1011da16b5c5ff721afbf0 This commit was not applied on nvd1/fkie fetcher. Signed-off-by: Peter Marko --- meta/recipes-core/meta/cve-update-db-native.bb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index fe7b8a017f..8a3746a9c1 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -58,7 +58,7 @@ python do_fetch() { bb.error("CVE database %s not present, database fetch/update skipped" % db_file) return if time.time() - os.path.getmtime(db_file) < update_interval: - bb.debug(2, "Recently updated, skipping") + bb.note("CVE database recently updated, skipping") return except OSError: @@ -77,7 +77,7 @@ python do_fetch() { shutil.move(db_tmp_file, db_file) else: # Update failed, do not modify the database - bb.note("CVE database update failed") + bb.warn("CVE database update failed") os.remove(db_tmp_file) } @@ -159,7 +159,7 @@ def update_db_file(db_tmp_file, d): with bb.progress.ProgressHandler(d) as ph, open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a') as cve_f: total_years = date.today().year + 1 - YEAR_START for i, year in enumerate(range(YEAR_START, date.today().year + 1)): - bb.debug(2, "Updating %d" % year) + bb.note("Updating %d" % year) ph.update((float(i + 1) / total_years) * 100) json_url, meta_url = db_file_names(d, year, is_nvd) @@ -190,7 +190,7 @@ def update_db_file(db_tmp_file, d): cursor.close() if not meta or meta[0] != last_modified: - bb.debug(2, "Updating entries") + bb.note("Updating entries") # Clear products table entries corresponding to current year conn.execute("delete from PRODUCTS where ID like ?", ('CVE-%d%%' % year,)).close() From patchwork Sun Aug 24 14:57:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Marko X-Patchwork-Id: 69089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F3D8CA0EEB for ; Sun, 24 Aug 2025 14:59:41 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.web10.18222.1756047572994020825 for ; Sun, 24 Aug 2025 07:59:33 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm2 header.b=X4IRYbzO; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-256628-2025082414593104b7252aee4973825b-yvjox_@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 2025082414593104b7252aee4973825b for ; Sun, 24 Aug 2025 16:59:31 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=kRwxljFGk/m2uU+0TPOWZ1EldkHAQRb94q7iZVqHGYE=; b=X4IRYbzO6vsCneDNHB7GbbelWdWUu864EIm+RqMCared6MGgpXg/2g2yQEdRhyFQ0akKPr 7DmZ0SP0MNGc324Vx+vP0/O5T+A2Q1psQHbhJJJKSFSEvKMziZlTl2zT9lC1zNbYe7XFd+jO lrhWF4Cw/e42HhMX0ocSwveSxizg1HbxEJAvGsUSO0m7E0aEzcbfbgNmOBIdOAC374l0Tkzk GnAmE7cfzqUhEU5w8Nv3+Pq0NbmzJBUkEsTTtZd8g2bfHdIVI3PVW0JfMNllQjqq6b2BAhJl 8iddQlp5J8B5aQ2Rp27Dzfm9jylSGIX6vvtTsMhT/YZjJdeyEAz2etnQ==; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Peter Marko Subject: [OE-core][PATCH 4/7] cve-update: decrease update interval to 23 hours Date: Sun, 24 Aug 2025 16:57:42 +0200 Message-Id: <20250824145745.1099373-4-peter.marko@siemens.com> In-Reply-To: <20250824145745.1099373-1-peter.marko@siemens.com> References: <20250824145745.1099373-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 24 Aug 2025 14:59:41 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/222384 From: Peter Marko If the job runs every day at the same time, it usually updates only every second day, because it takes non-0 time for DB update and set the timestamp. So it does not take full 24-hours from time when the DB was updated until the next job starts. Signed-off-by: Peter Marko --- meta/recipes-core/meta/cve-update-db-native.bb | 4 ++-- meta/recipes-core/meta/cve-update-nvd2-native.bb | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index 8a3746a9c1..ecdb1ed8fd 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -14,10 +14,10 @@ deltask do_populate_sysroot NVDCVE_URL ?= "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-" FKIE_URL ?= "https://github.com/fkie-cad/nvd-json-data-feeds/releases/latest/download/CVE-" -# CVE database update interval, in seconds. By default: once a day (24*60*60). +# CVE database update interval, in seconds. By default: once a day (23*60*60). # Use 0 to force the update # Use a negative value to skip the update -CVE_DB_UPDATE_INTERVAL ?= "86400" +CVE_DB_UPDATE_INTERVAL ?= "82800" # Timeout for blocking socket operations, such as the connection attempt. CVE_SOCKET_TIMEOUT ?= "60" diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 32a14a932b..83876c7467 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -20,10 +20,10 @@ NVDCVE_URL ?= "https://services.nvd.nist.gov/rest/json/cves/2.0" # then setting this to get higher rate limits. NVDCVE_API_KEY ?= "" -# CVE database update interval, in seconds. By default: once a day (24*60*60). +# CVE database update interval, in seconds. By default: once a day (23*60*60). # Use 0 to force the update # Use a negative value to skip the update -CVE_DB_UPDATE_INTERVAL ?= "86400" +CVE_DB_UPDATE_INTERVAL ?= "82800" # CVE database incremental update age threshold, in seconds. If the database is # older than this threshold, do a full re-download, else, do an incremental From patchwork Sun Aug 24 14:57:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Marko X-Patchwork-Id: 69090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3CEDCA0EFF for ; Sun, 24 Aug 2025 14:59:41 +0000 (UTC) Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net [185.136.65.228]) by mx.groups.io with SMTP id smtpd.web11.18374.1756047580918115640 for ; Sun, 24 Aug 2025 07:59:41 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm2 header.b=HWDALIZn; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.228, mailfrom: fm-256628-2025082414593817a906d5d9bd18d836-yaop_2@rts-flowmailer.siemens.com) Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 2025082414593817a906d5d9bd18d836 for ; Sun, 24 Aug 2025 16:59:38 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=y7g0AQY1yBtG4zUn3ezNtKWHjMSxAUz1pUJMwzlMUNk=; b=HWDALIZn2zWfUfAAWPVR7/qlzjPKBD4IwFam1RpQDVv7KsROuhttTRXbyx4NWmCzQ/OFbj YfzCMuVYHYJPO5Vay26megvQuA2OC1EnSHSI1o+I5njLnp3CCpjRG2Sh13xRVnKJRi2oFdIv nPZBGDujSMsV2r8fEXs1ocvIPUlmHiMvseXkwdTIFrkTfSNsbH/fj21VGoYv97pI6zlPAGer KPh7Kq9VhellLMWYx7cAmp7B4bWy43fTGXEJG7UkTxNpyjTkekgwUc0bI0XfOPVuDgz37z8k rubowroESqFlfeUHGulR8SKA4C5JzC/PAa1ed19RE+RGHJJudyO87WfA==; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Peter Marko Subject: [OE-core][PATCH 5/7] cve-update: remove cleanup of db_file in downloads Date: Sun, 24 Aug 2025 16:57:43 +0200 Message-Id: <20250824145745.1099373-5-peter.marko@siemens.com> In-Reply-To: <20250824145745.1099373-1-peter.marko@siemens.com> References: <20250824145745.1099373-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 24 Aug 2025 14:59:41 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/222385 From: Peter Marko Since the code was changed to update the DB in temporary file, code cleaning the final file in downloads is never executed. Remove it. Since the code always removes both files in temporary directory, remove also comment which is trying to differentiate this code from code just removed. Signed-off-by: Peter Marko --- meta/recipes-core/meta/cve-update-db-native.bb | 17 ++--------------- .../recipes-core/meta/cve-update-nvd2-native.bb | 17 ++--------------- 2 files changed, 4 insertions(+), 30 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index ecdb1ed8fd..43cafb52b1 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -45,7 +45,7 @@ python do_fetch() { db_dir = os.path.dirname(db_file) db_tmp_file = d.getVar("CVE_CHECK_DB_TEMP_FILE") - cleanup_db_download(db_file, db_tmp_file) + cleanup_db_download(db_tmp_file) # The NVD database changes once a day, so no need to update more frequently # Allow the user to force-update @@ -91,28 +91,15 @@ python do_unpack() { } do_unpack[lockfiles] += "${CVE_CHECK_DB_DLDIR_LOCK} ${CVE_CHECK_DB_FILE_LOCK}" -def cleanup_db_download(db_file, db_tmp_file): +def cleanup_db_download(db_tmp_file): """ Cleanup the download space from possible failed downloads """ - # Clean up the updates done on the main file - # Remove it only if a journal file exists - it means a complete re-download - if os.path.exists("{0}-journal".format(db_file)): - # If a journal is present the last update might have been interrupted. In that case, - # just wipe any leftovers and force the DB to be recreated. - os.remove("{0}-journal".format(db_file)) - - if os.path.exists(db_file): - os.remove(db_file) - # Clean-up the temporary file downloads, we can remove both journal # and the temporary database if os.path.exists("{0}-journal".format(db_tmp_file)): - # If a journal is present the last update might have been interrupted. In that case, - # just wipe any leftovers and force the DB to be recreated. os.remove("{0}-journal".format(db_tmp_file)) - if os.path.exists(db_tmp_file): os.remove(db_tmp_file) diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 83876c7467..f7a306c995 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -57,7 +57,7 @@ python do_fetch() { db_dir = os.path.dirname(db_file) db_tmp_file = d.getVar("CVE_CHECK_DB_TEMP_FILE") - cleanup_db_download(db_file, db_tmp_file) + cleanup_db_download(db_tmp_file) # By default let's update the whole database (since time 0) database_time = 0 @@ -106,28 +106,15 @@ python do_unpack() { } do_unpack[lockfiles] += "${CVE_CHECK_DB_DLDIR_LOCK} ${CVE_CHECK_DB_FILE_LOCK}" -def cleanup_db_download(db_file, db_tmp_file): +def cleanup_db_download(db_tmp_file): """ Cleanup the download space from possible failed downloads """ - # Clean up the updates done on the main file - # Remove it only if a journal file exists - it means a complete re-download - if os.path.exists("{0}-journal".format(db_file)): - # If a journal is present the last update might have been interrupted. In that case, - # just wipe any leftovers and force the DB to be recreated. - os.remove("{0}-journal".format(db_file)) - - if os.path.exists(db_file): - os.remove(db_file) - # Clean-up the temporary file downloads, we can remove both journal # and the temporary database if os.path.exists("{0}-journal".format(db_tmp_file)): - # If a journal is present the last update might have been interrupted. In that case, - # just wipe any leftovers and force the DB to be recreated. os.remove("{0}-journal".format(db_tmp_file)) - if os.path.exists(db_tmp_file): os.remove(db_tmp_file) From patchwork Sun Aug 24 14:57:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Marko X-Patchwork-Id: 69091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DC7ECA0EEB for ; Sun, 24 Aug 2025 14:59:51 +0000 (UTC) Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net [185.136.65.228]) by mx.groups.io with SMTP id smtpd.web10.18231.1756047588303922273 for ; Sun, 24 Aug 2025 07:59:48 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm2 header.b=IliQf9Lp; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.228, mailfrom: fm-256628-202508241459461c6a3b011e14225d88-qkfhir@rts-flowmailer.siemens.com) Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 202508241459461c6a3b011e14225d88 for ; Sun, 24 Aug 2025 16:59:46 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=D1lCcwPX+E0QV/t/QtPiM4HpbTQ1oOCc4Iocf3sZrms=; b=IliQf9LptoQZqwT0nz1aK0ynjv4oDL8mRvx4TIVIVqutuGoGai50idvGOueD9V4l5NPthr caz0Nb/Z/M3MtfNMvOczhgWfEE0iN7VcwoMiGIe44fDsQmXODvR02VcwOuxYsKDBKd4TL7O7 h/p5VcEjkRHPERZL2mu+0WzuQ+77FOEprWUTgCSiAPd6LKl2L8g3eC3fkmrA1+jpA1UW7KsJ owG7WWuCbA+0qzAVM3kwnFwqaw372UyyJu1vPliyqezSKPbEj0n4bglZLK1FScOwHNnzhRu5 /oCtIc/CRWb+uuJ00ydXTZO8rTVmbam1AqafMbdRpOLmZvypy7biPlzQ==; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Jonathan Schnitzler , Peter Marko Subject: [OE-core][PATCH 6/7] cve-update-db-native: Fix FKIE CVE accessVector parsing Date: Sun, 24 Aug 2025 16:57:44 +0200 Message-Id: <20250824145745.1099373-6-peter.marko@siemens.com> In-Reply-To: <20250824145745.1099373-1-peter.marko@siemens.com> References: <20250824145745.1099373-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 24 Aug 2025 14:59:51 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/222386 From: Jonathan Schnitzler Use "attackVector" for CVSS >= 3 as it only CVSS v2 uses "accessVector". Signed-off-by: Jonathan Schnitzler Signed-off-by: Peter Marko --- meta/recipes-core/meta/cve-update-db-native.bb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index 43cafb52b1..87e5a3edaa 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -352,7 +352,7 @@ def update_db_fkie(conn, jsondata): try: for m in elt['metrics']['cvssMetricV30']: if m['type'] == 'Primary': - accessVector = m['cvssData']['accessVector'] + accessVector = m['cvssData']['attackVector'] vectorString = m['cvssData']['vectorString'] cvssv3 = m['cvssData']['baseScore'] except KeyError: @@ -361,7 +361,7 @@ def update_db_fkie(conn, jsondata): try: for m in elt['metrics']['cvssMetricV31']: if m['type'] == 'Primary': - accessVector = m['cvssData']['accessVector'] + accessVector = m['cvssData']['attackVector'] vectorString = m['cvssData']['vectorString'] cvssv3 = m['cvssData']['baseScore'] except KeyError: @@ -370,7 +370,7 @@ def update_db_fkie(conn, jsondata): try: for m in elt['metrics']['cvssMetricV40']: if m['type'] == 'Primary': - accessVector = m['cvssData']['accessVector'] + accessVector = m['cvssData']['attackVector'] vectorString = m['cvssData']['vectorString'] cvssv4 = m['cvssData']['baseScore'] except KeyError: From patchwork Sun Aug 24 14:57:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Marko X-Patchwork-Id: 69092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7B71CA0EEB for ; Sun, 24 Aug 2025 15:00:01 +0000 (UTC) Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net [185.136.65.227]) by mx.groups.io with SMTP id smtpd.web10.18233.1756047596243005388 for ; Sun, 24 Aug 2025 07:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm2 header.b=brpKmqB3; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.227, mailfrom: fm-256628-2025082414595430534d23a4bbb54eb7-1ckjc7@rts-flowmailer.siemens.com) Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 2025082414595430534d23a4bbb54eb7 for ; Sun, 24 Aug 2025 16:59:54 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=61zZV1ov0EfOSa7NcYfA8T4UHxjN/WEMsLO0cZtqbF4=; b=brpKmqB3IujdrDWdwfQrZ0HltGKsxEp1RHYwsU10Z3zzQ7F3Ly58IyUSEBnlogxghUeBF6 adOZyQPkHjCrv/tJoRD4wlYyb7ZHAm2AXVDysHEiXhsixQdh5mhLzurFornl3EBBcsTWJIj+ maR4A5+uaqMk7TpsRgnTQ5Z7cHLRW7R6Vj9HGKyntJHkuuVTIhei/LYMCWTF8U75OPFg5ve8 CzvCJsgPqOLpFsfVGRqpOkG7KxE9kTx5ubxbEjqai7HigNKaTxOrBdjIyDvoCpFtLGkQsTeu rHF1M+pJAX6Jt7saM6md47vtYlO956mPryhKq9PmGtV26d7N9OJZ2yOw==; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Jonathan Schnitzler , Peter Marko Subject: [OE-core][PATCH 7/7] cve-update-db-native: FKIE CVE parsing: Use Secondary metric Date: Sun, 24 Aug 2025 16:57:45 +0200 Message-Id: <20250824145745.1099373-7-peter.marko@siemens.com> In-Reply-To: <20250824145745.1099373-1-peter.marko@siemens.com> References: <20250824145745.1099373-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 24 Aug 2025 15:00:01 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/222387 From: Jonathan Schnitzler If there is no primary metric use the Secondary one. Signed-off-by: Jonathan Schnitzler Signed-off-by: Peter Marko --- .../recipes-core/meta/cve-update-db-native.bb | 53 ++++++++++++------- 1 file changed, 33 insertions(+), 20 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index 87e5a3edaa..0c7bc5f415 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -322,6 +322,15 @@ def update_db_nvdjson(conn, jsondata): for config in configurations: parse_node_and_insert(conn, config, cveId, True) +def get_metric_entry(metric): + primaries = [c for c in metric if c['type'] == "Primary"] + secondaries = [c for c in metric if c['type'] == "Secondary"] + if len(primaries) > 0: + return primaries[0] + elif len(secondaries)>0: + return secondaries[0] + return None + def update_db_fkie(conn, jsondata): import json root = json.loads(jsondata) @@ -342,37 +351,41 @@ def update_db_fkie(conn, jsondata): cveDesc = elt['descriptions'][0]['value'] date = elt['lastModified'] try: - for m in elt['metrics']['cvssMetricV2']: - if m['type'] == 'Primary': - accessVector = m['cvssData']['accessVector'] - vectorString = m['cvssData']['vectorString'] - cvssv2 = m['cvssData']['baseScore'] + if 'cvssMetricV2' in elt['metrics']: + entry = get_metric_entry(elt['metrics']['cvssMetricV2']) + if entry: + accessVector = entry['cvssData']['accessVector'] + vectorString = entry['cvssData']['vectorString'] + cvssv2 = entry['cvssData']['baseScore'] except KeyError: cvssv2 = 0.0 try: - for m in elt['metrics']['cvssMetricV30']: - if m['type'] == 'Primary': - accessVector = m['cvssData']['attackVector'] - vectorString = m['cvssData']['vectorString'] - cvssv3 = m['cvssData']['baseScore'] + if 'cvssMetricV30' in elt['metrics']: + entry = get_metric_entry(elt['metrics']['cvssMetricV30']) + if entry: + accessVector = entry['cvssData']['attackVector'] + vectorString = entry['cvssData']['vectorString'] + cvssv3 = entry['cvssData']['baseScore'] except KeyError: accessVector = accessVector or "UNKNOWN" cvssv3 = 0.0 try: - for m in elt['metrics']['cvssMetricV31']: - if m['type'] == 'Primary': - accessVector = m['cvssData']['attackVector'] - vectorString = m['cvssData']['vectorString'] - cvssv3 = m['cvssData']['baseScore'] + if 'cvssMetricV31' in elt['metrics']: + entry = get_metric_entry(elt['metrics']['cvssMetricV31']) + if entry: + accessVector = entry['cvssData']['attackVector'] + vectorString = entry['cvssData']['vectorString'] + cvssv3 = entry['cvssData']['baseScore'] except KeyError: accessVector = accessVector or "UNKNOWN" cvssv3 = 0.0 try: - for m in elt['metrics']['cvssMetricV40']: - if m['type'] == 'Primary': - accessVector = m['cvssData']['attackVector'] - vectorString = m['cvssData']['vectorString'] - cvssv4 = m['cvssData']['baseScore'] + if 'cvssMetricV40' in elt['metrics']: + entry = get_metric_entry(elt['metrics']['cvssMetricV40']) + if entry: + accessVector = entry['cvssData']['attackVector'] + vectorString = entry['cvssData']['vectorString'] + cvssv4 = entry['cvssData']['baseScore'] except KeyError: accessVector = accessVector or "UNKNOWN" cvssv4 = 0.0