| Message ID | 20260116190520.118714-1-valentin.boudevin@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/1] improve_kerne_cve_report: Add a bbclass support | expand |
Hi Valentin, Thanks for your contribution. A bit more feedback. There is a typo in the patch subject kerne/kernel > -----Original Message----- > From: ValentinBoudevin <valentin.boudevin@gmail.com> > Sent: Friday, 16 January 2026 20:05 > To: openembedded-core@lists.openembedded.org > Cc: Daniel Turull <daniel.turull@ericsson.com>; > jerome.oufella@savoirfairelinux.com; antonin.godard@bootlin.com; > ValentinBoudevin <valentin.boudevin@gmail.com> > Subject: [PATCH 1/1] improve_kerne_cve_report: Add a bbclass support > > The script improve_kernel_cve_report.py doesn't have a bbclass. > It can be usefull to have one to generate improved cve-check files at every > run. > > This new class can be used to generate a new file in tmp/deploy/images with > a .scouted.json in addition to the existing .json cve-check file. > > The new .scouted.json is based on the cve-check file and the SBOM (SPDX3 > mandatory) to generate this improved cve-check file with extra entries found > by the script improve_kernel_cve_report.py. > > It only requires an inherit on an image recipe (e.g. "inherit > improve_kernel_cve_report" in core-image-minimal). > > It can be add to core-image-minimal in a second step if revelant. > > Signed-off-by: Valentin Boudevin <valentin.boudevin@gmail.com> > --- > .../classes/improve_kernel_cve_report.bbclass | 71 +++++++++++++++++++ > 1 file changed, 71 insertions(+) > create mode 100644 meta/classes/improve_kernel_cve_report.bbclass > > diff --git a/meta/classes/improve_kernel_cve_report.bbclass > b/meta/classes/improve_kernel_cve_report.bbclass > new file mode 100644 > index 0000000000..5c496252b4 > --- /dev/null > +++ b/meta/classes/improve_kernel_cve_report.bbclass > @@ -0,0 +1,71 @@ > +python do_clean:append() { > + import os, glob > + if bb.utils.contains('INHERIT', 'create-spdx-2.2', 'false', 'true', d): > + deploy_dir = d.expand('${DEPLOY_DIR_IMAGE}') > + for f in glob.glob(os.path.join(deploy_dir, '*scouted.json')): > + bb.note("Removing " + f) > + os.remove(f) > +} > + > +python do_clone_kernel_cve() { > + import subprocess > + import shutil, os > + check_spdx = d.getVar("INHERIT") > + rootdir = os.path.join(d.getVar("WORKDIR"), "vulns") > + # Check if the feature is enabled and if SPDX 2.2 is not used > + if "create-spdx-2.2" not in check_spdx: > + d.setVar("SRC_URI", > "git://git.kernel.org/pub/scm/linux/security/vulns.git;branch=master;protocol > =https") > + d.setVar("SRCREV", "${AUTOREV}") > + src_uri = (d.getVar('SRC_URI') or "").split() > + # Fetch the kernel vulnerabilities sources > + fetcher = bb.fetch2.Fetch(src_uri, d) > + fetcher.download() > + # Unpack into the standard work directory > + fetcher.unpack(rootdir) > + # Remove the folder ${PN} set by unpack > + subdirs = [d for d in os.listdir(rootdir) if > os.path.isdir(os.path.join(rootdir, d))] > + if len(subdirs) == 1: > + srcdir = os.path.join(rootdir, subdirs[0]) > + for f in os.listdir(srcdir): > + shutil.move(os.path.join(srcdir, f), rootdir) > + shutil.rmtree(srcdir) > + bb.note("Vulnerabilities repo unpacked into: %s" % rootdir) > + elif "create-spdx-2.2" in check_spdx: > + bb.warn(f"improve_kernel_cve_report: Extra Kernel CVEs Scouting > +is desactivate because incompatible with SPDX 2.2.") } The script accepts both spdx2 and spdx3 > +do_clone_kernel_cve[network] = "1" > +do_clone_kernel_cve[nostamp] = "1" > +do_clone_kernel_cve[doc] = "Clone the latest kernel vulnerabilities from > https://git.kern/ > el.org%2Fpub%2Fscm%2Flinux%2Fsecurity%2Fvulns.git&data=05%7C02%7Cd > aniel.turull%40ericsson.com%7Cca7acecb518d48391e3408de55333e8e%7C92 > e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C639041875752180972%7C > Unknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA > wMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C% > 7C&sdata=EXS18CY6gFkHdj4C%2F6%2BeknzXrU8h4b3Uct8T2OAQTFE%3D&re > served=0" > +addtask clone_kernel_cve after > + > +do_scout_extra_kernel_vulns() { > + spdx_file="${SPDXIMAGEDEPLOYDIR}/${IMAGE_LINK_NAME}.spdx.json" > + > original_cve_check_file="${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.json > " > + > new_cve_report_file="${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.scouted.json > " > + > improve_kernel_cve_script="${COREBASE}/scripts/contrib/improve_kernel_cv > e_report.py" > + > + if ${@bb.utils.contains('INHERIT', 'create-spdx-2.2', 'true', 'false', d)}; then > + bbwarn "improve_kernel_cve_report: Skipping extra kernel > vulnerabilities scouting because incompatible with SPDX 2." > + return 0 > + elif [ ! -f "${spdx_file}" ]; then > + bbwarn "improve_kernel_cve_report: SPDX file not found: ${spdx_file}. > Skipping extra kernel vulnerabilities scoutings." > + return 0 > + elif [ ! -f "${original_cve_check_file}" ]; then > + bbwarn "improve_kernel_cve_report: CVE_CHECK file not found: > ${original_cve_check_file}. Skipping extra kernel vulnerabilities scouting." > + return 0 > + fi > + > + #Launch the new script to improve the cve report > + python3 "${improve_kernel_cve_script}" \ > + --spdx "${spdx_file}" \ The script automatically detects the spdx format. You can also use the debugsources file to be spdx independent. > + --old-cve-report "${original_cve_check_file}" \ > + --new-cve-report "${new_cve_report_file}" \ > + --datadir "${WORKDIR}/vulns" > + bbplain "Improve CVE report with extra kernel cves: > ${new_cve_report_file}" > + > + #Create a symlink as every other JSON file in tmp/deploy/images > + ln -sf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.scouted.json > +${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}$ > {IMAGE_NAM > +E_SUFFIX}.scouted.json > +} > +do_scout_extra_kernel_vulns[nostamp] = "1" > +do_scout_extra_kernel_vulns[doc] = "Scout extra kernel vulnerabilities and > create a new enhanced version of the cve_check file in the deploy directory" > +addtask scout_extra_kernel_vulns after do_create_image_sbom_spdx > before > +do_build > \ No newline at end of file Daniel
Thanks for the changes Valentin,
I have tested series 5 as offline build. When it is not enable, it doesn't interfere with the rest of the build.
For offline build, I had to setup a fix GENERATE_CVE_EXCLUSIONS_SRCREV, since the fetcher doesn't work with AUTOREV (kind of expected). This seems to work.
ENABLE_KERNEL_CVE_EXCLUSIONS = "1"
GENERATE_CVE_EXCLUSIONS_SRCREV = "bcd089af283df7902b33e9cf0cedef5aa2c4a298"
GENERATE_CVE_EXCLUSIONS_NETWORK = "0"
When setting GENERATE_CVE_EXCLUSIONS_SRC_URI to an internal mirror it also works with AUTOREV
ENABLE_KERNEL_CVE_EXCLUSIONS = "1"
GENERATE_CVE_EXCLUSIONS_SRC_URI = "git://internal-mirror/github.com.CVEProject.cvelistV5;protocol=https;branch=main"
GENERATE_CVE_EXCLUSIONS_SRCREV = "AUTOREV"
I have also look at the generate SPDX for the linux-yocto, it includes an additional source, which technically is not that correct I think
jq . tmp/deploy/spdx/3.0.1/qemuarm64/recipes/recipe-linux-yocto.spdx.json | grep cvelistV5 -A 3 -B 10
{
"type": "software_Package",
"spdxId": "http://spdx.org/spdxdocs/linux-yocto-f9f75dbe-e63f-5a48-86d8-e19d0ec693db/a35fa1d8530e46c84d59b6776858eafc75bdd98b8439c2ee06db6fdab151a8c3/source/3",
"creationInfo": "_:CreationInfo0",
"extension": [
{
"type": "https://rdf.openembedded.org/spdx/3.0/id-alias",
"https://rdf.openembedded.org/spdx/3.0/alias": "http://spdxdocs.org/openembedded-alias/by-doc-hash/8fe80285f43eb235d61996ced46668293eb3feaba85810b14aed822b4680e56f/linux-yocto/UNIHASH/source/3"
}
],
"name": "github.com.CVEProject.cvelistV5.git",
"software_primaryPurpose": "source",
"software_downloadLocation": "git+https://github.com/CVEProject/cvelistV5.git@bcd089af283df7902b33e9cf0cedef5aa2c4a298"
},
Also bitbake linux-yocto -c do_generate_cve_exclusions works as expected, if it will be updating the inc file that is used in the linux-yocto with the exclusions, these will also show up in the SPDX file.
Thanks again for trying to push the changes that you have in vulnscout into oe-core.
Best regards
Daniel
> -----Original Message-----
> From: ValentinBoudevin <valentin.boudevin@gmail.com>
> Sent: Friday, 16 January 2026 20:05
> To: openembedded-core@lists.openembedded.org
> Cc: Daniel Turull <daniel.turull@ericsson.com>;
> jerome.oufella@savoirfairelinux.com; antonin.godard@bootlin.com;
> ValentinBoudevin <valentin.boudevin@gmail.com>
> Subject: [PATCH v5 0/4] generate-cve-exclusions: Add a .bbclass
>
> Changes since v4:
> - Patch 2/4:
> * Renamed the bbclass to kernel-generate-cve-exclusions.bbclass to better
> reflect its purpose.
> * Add new variable ENABLE_KERNEL_CVE_EXCLUSIONS to enable/disable
> the
> feature.
> By default, the feature is disabled to avoid unexpected behavior on
> existing builds with linux-yocto.
> * Add new "__anonymous" python function to setup the variables SRC_URI
> and SRCREV only if
> this feature is enabled with ENABLE_KERNEL_CVE_EXCLUSIONS.
> Also prevent from modifying SRC_URI and SRCREV variables in the default
> linux-yocto usecase.
> Now, the recipe does not have any impact on the basic "linux-yocto" recipe if
> the feature is disabled.
> * Add new variables GENERATE_CVE_EXCLUSIONS_DESTSUFFIX and
> GENERATE_CVE_EXCLUSIONS_UNPACK_DIR to customize the working
> directory path of the
> class.
> - Patch 4/4:
> * Update the inherit statement in linux-yocto.inc to reflect the new name of
> the bbclass with
> "kernel-generate-cve-exclusions".
>
> Changes since v3:
> - Patch 2/4:
> * Add variables to control offline mode, source URI and
> SRCREV for deterministic testing (GENERATE_CVE_EXCLUSIONS_SRC_URI,
> GENERATE_CVE_EXCLUSIONS_SRCREV,
> GENERATE_CVE_EXCLUSIONS_NETWORK).
> * Updated generate_cve_exclusions task scheduling to be executed before
> do_cve_check.
>
> Changes since v2:
> - Patch 4/4: Inherit the new bbclass in linux-yocto.inc instead of
> individual recipes.
>
> Changes since v1:
> - Patch 2/4: Removed the mandatory execution of the
> generate-cve-exclusions class on every build. It now needs to be
> manually run using:
> bitbake -c generate-cve-exclusions <kernel-recipe>
>
> ValentinBoudevin (4):
> generate-cve-exclusions: Add --output-json option
> generate-cve-exclusions: Add a .bbclass
> generate-cve-exclusions: Move python script
> linux: Add inherit on generate-cve-exclusions
>
> .../kernel-generate-cve-exclusions.bbclass | 135 ++++++++++++++++++
> meta/recipes-kernel/linux/linux-yocto.inc | 3 +
> .../contrib}/generate-cve-exclusions.py | 64 +++++++--
> 3 files changed, 188 insertions(+), 14 deletions(-) create mode 100644
> meta/classes/kernel-generate-cve-exclusions.bbclass
> rename {meta/recipes-kernel/linux => scripts/contrib}/generate-cve-
> exclusions.py (71%)
diff --git a/meta/classes/improve_kernel_cve_report.bbclass b/meta/classes/improve_kernel_cve_report.bbclass new file mode 100644 index 0000000000..5c496252b4 --- /dev/null +++ b/meta/classes/improve_kernel_cve_report.bbclass @@ -0,0 +1,71 @@ +python do_clean:append() { + import os, glob + if bb.utils.contains('INHERIT', 'create-spdx-2.2', 'false', 'true', d): + deploy_dir = d.expand('${DEPLOY_DIR_IMAGE}') + for f in glob.glob(os.path.join(deploy_dir, '*scouted.json')): + bb.note("Removing " + f) + os.remove(f) +} + +python do_clone_kernel_cve() { + import subprocess + import shutil, os + check_spdx = d.getVar("INHERIT") + rootdir = os.path.join(d.getVar("WORKDIR"), "vulns") + # Check if the feature is enabled and if SPDX 2.2 is not used + if "create-spdx-2.2" not in check_spdx: + d.setVar("SRC_URI", "git://git.kernel.org/pub/scm/linux/security/vulns.git;branch=master;protocol=https") + d.setVar("SRCREV", "${AUTOREV}") + src_uri = (d.getVar('SRC_URI') or "").split() + # Fetch the kernel vulnerabilities sources + fetcher = bb.fetch2.Fetch(src_uri, d) + fetcher.download() + # Unpack into the standard work directory + fetcher.unpack(rootdir) + # Remove the folder ${PN} set by unpack + subdirs = [d for d in os.listdir(rootdir) if os.path.isdir(os.path.join(rootdir, d))] + if len(subdirs) == 1: + srcdir = os.path.join(rootdir, subdirs[0]) + for f in os.listdir(srcdir): + shutil.move(os.path.join(srcdir, f), rootdir) + shutil.rmtree(srcdir) + bb.note("Vulnerabilities repo unpacked into: %s" % rootdir) + elif "create-spdx-2.2" in check_spdx: + bb.warn(f"improve_kernel_cve_report: Extra Kernel CVEs Scouting is desactivate because incompatible with SPDX 2.2.") +} +do_clone_kernel_cve[network] = "1" +do_clone_kernel_cve[nostamp] = "1" +do_clone_kernel_cve[doc] = "Clone the latest kernel vulnerabilities from https://git.kernel.org/pub/scm/linux/security/vulns.git" +addtask clone_kernel_cve after + +do_scout_extra_kernel_vulns() { + spdx_file="${SPDXIMAGEDEPLOYDIR}/${IMAGE_LINK_NAME}.spdx.json" + original_cve_check_file="${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.json" + new_cve_report_file="${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.scouted.json" + improve_kernel_cve_script="${COREBASE}/scripts/contrib/improve_kernel_cve_report.py" + + if ${@bb.utils.contains('INHERIT', 'create-spdx-2.2', 'true', 'false', d)}; then + bbwarn "improve_kernel_cve_report: Skipping extra kernel vulnerabilities scouting because incompatible with SPDX 2." + return 0 + elif [ ! -f "${spdx_file}" ]; then + bbwarn "improve_kernel_cve_report: SPDX file not found: ${spdx_file}. Skipping extra kernel vulnerabilities scoutings." + return 0 + elif [ ! -f "${original_cve_check_file}" ]; then + bbwarn "improve_kernel_cve_report: CVE_CHECK file not found: ${original_cve_check_file}. Skipping extra kernel vulnerabilities scouting." + return 0 + fi + + #Launch the new script to improve the cve report + python3 "${improve_kernel_cve_script}" \ + --spdx "${spdx_file}" \ + --old-cve-report "${original_cve_check_file}" \ + --new-cve-report "${new_cve_report_file}" \ + --datadir "${WORKDIR}/vulns" + bbplain "Improve CVE report with extra kernel cves: ${new_cve_report_file}" + + #Create a symlink as every other JSON file in tmp/deploy/images + ln -sf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.scouted.json ${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}${IMAGE_NAME_SUFFIX}.scouted.json +} +do_scout_extra_kernel_vulns[nostamp] = "1" +do_scout_extra_kernel_vulns[doc] = "Scout extra kernel vulnerabilities and create a new enhanced version of the cve_check file in the deploy directory" +addtask scout_extra_kernel_vulns after do_create_image_sbom_spdx before do_build \ No newline at end of file
The script improve_kernel_cve_report.py doesn't have a bbclass. It can be usefull to have one to generate improved cve-check files at every run. This new class can be used to generate a new file in tmp/deploy/images with a .scouted.json in addition to the existing .json cve-check file. The new .scouted.json is based on the cve-check file and the SBOM (SPDX3 mandatory) to generate this improved cve-check file with extra entries found by the script improve_kernel_cve_report.py. It only requires an inherit on an image recipe (e.g. "inherit improve_kernel_cve_report" in core-image-minimal). It can be add to core-image-minimal in a second step if revelant. Signed-off-by: Valentin Boudevin <valentin.boudevin@gmail.com> --- .../classes/improve_kernel_cve_report.bbclass | 71 +++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 meta/classes/improve_kernel_cve_report.bbclass