From patchwork Mon Nov 25 08:14:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 53092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43CB9D3B7F1 for ; Mon, 25 Nov 2024 08:15:12 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web11.14552.1732522504090782885 for ; Mon, 25 Nov 2024 00:15:04 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=1059dd1ea8=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4AP76maP031692; Mon, 25 Nov 2024 08:15:02 GMT Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [147.11.82.252]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4336189qve-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Mon, 25 Nov 2024 08:15:02 +0000 (GMT) Received: from ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.43; Mon, 25 Nov 2024 00:15:00 -0800 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server id 15.1.2507.43 via Frontend Transport; Mon, 25 Nov 2024 00:15:00 -0800 From: Hongxu Jia To: , Subject: [oe-core][PATCH V3 1/3] sbom30/spdx30: add link prefix to the namespace of spdxId and alias Date: Mon, 25 Nov 2024 00:14:58 -0800 Message-ID: <20241125081500.4048680-2-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241125081500.4048680-1-hongxu.jia@windriver.com> References: <20241125081500.4048680-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: FhhzZivhFhRl7tSB9h_zD-_Ken43eDgh X-Proofpoint-GUID: FhhzZivhFhRl7tSB9h_zD-_Ken43eDgh X-Authority-Analysis: v=2.4 cv=O65rvw9W c=1 sm=1 tr=0 ts=67443206 cx=c_pps a=/ZJR302f846pc/tyiSlYyQ==:117 a=/ZJR302f846pc/tyiSlYyQ==:17 a=VlfZXiiP6vEA:10 a=24AZYWMyAAAA:8 a=t7CeM3EgAAAA:8 a=M7lxqswjPBWPblf4S0MA:9 a=bG88sKzkDEFeXWNnvthB:22 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-25_05,2024-11-21_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 clxscore=1015 mlxscore=0 bulkscore=0 mlxlogscore=999 spamscore=0 priorityscore=1501 lowpriorityscore=0 adultscore=0 malwarescore=0 impostorscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2411250069 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Nov 2024 08:15:12 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/207723 In order to simple reference the SPDX ID to instead of making jsonld hash path for each element, only creating one symlink for one file and referencing it multiple times, add link prefix and name to the namespace of spdxId and alias to replace ${PN} to avoid namespace conflict between recipe, packages and images. Take recipe shadow, package shadow and package shadow-src for example: Without this commit, spdxId and alias in recipe and package jsonld have the same namespace spdxId: http://spdx.org/spdxdocs/shadow-xxx/... alias: http://spdx.org/spdxdocs/openembedded-alias/shadow/UNIHASH/... After apply this commit, the namespace of spdxId in recipe and package jsonld differs: In recipe jsonld tmp/deploy/spdx/3.0.1/core2-64/recipes/shadow.spdx.json spdxId: http://spdx.org/spdxdocs/recipe-shadow-xxx/... alias: http://spdx.org/spdxdocs/openembedded-alias/recipe-shadow/UNIHASH/... In package jsonld tmp/deploy/spdx/3.0.1/core2-64/packages/shadow.spdx.json spdxId: http://spdx.org/spdxdocs/package-shadow-xxx/... alias: http://spdx.org/spdxdocs/openembedded-alias/package-shadow/UNIHASH/... In package jsonld tmp/deploy/spdx/3.0.1/core2-64/packages/shadow-src.spdx.json spdxId: http://spdx.org/spdxdocs/package-shadow-src-xxx/... alias: http://spdx.org/spdxdocs/openembedded-alias/package-shadow-src/UNIHASH/... Then will use namespace of spdxId and alias to create link for jsonld file, one symlink for one jsonld file, referenced by elements multiple times Signed-off-by: Hongxu Jia --- meta/lib/oe/sbom30.py | 29 ++++++++++++++++++----------- meta/lib/oe/spdx30_tasks.py | 13 +++++++------ 2 files changed, 25 insertions(+), 17 deletions(-) diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py index 0a7b4c05fb..28d251a7ac 100644 --- a/meta/lib/oe/sbom30.py +++ b/meta/lib/oe/sbom30.py @@ -217,9 +217,11 @@ def to_list(l): class ObjectSet(oe.spdx30.SHACLObjectSet): - def __init__(self, d): + def __init__(self, d, name=None, link_prefix=None): super().__init__() self.d = d + self.name = name + self.link_prefix = link_prefix def create_index(self): self.by_sha256_hash = {} @@ -322,6 +324,8 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): uuid.NAMESPACE_DNS, self.d.getVar("SPDX_UUID_NAMESPACE") ) pn = self.d.getVar("PN") + if self.link_prefix and self.name: + pn = "%s-%s" % (self.link_prefix, self.name) return "%s/%s-%s" % ( self.d.getVar("SPDX_NAMESPACE_PREFIX"), pn, @@ -341,12 +345,15 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): elif namespace not in e._id: bb.warn(f"Namespace {namespace} not found in {e._id}") else: + pn = self.d.getVar("PN") + if self.link_prefix and self.name: + pn = "%s-%s" % (self.link_prefix, self.name) alias_ext = set_alias( e, e._id.replace(unihash, "UNIHASH").replace( namespace, - "http://spdx.org/spdxdocs/openembedded-alias/" - + self.d.getVar("PN"), + f"{self.d.getVar('SPDX_NAMESPACE_PREFIX')}/openembedded-alias/" + + pn, ), ) @@ -805,8 +812,8 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): ) @classmethod - def new_objset(cls, d, name, copy_from_bitbake_doc=True): - objset = cls(d) + def new_objset(cls, d, name, copy_from_bitbake_doc=True, link_prefix=None): + objset = cls(d, name=name, link_prefix=link_prefix) document = oe.spdx30.SpdxDocument( _id=objset.new_spdxid("document", name), @@ -887,9 +894,9 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): return missing_spdxids -def load_jsonld(d, path, required=False): +def load_jsonld(d, path, required=False, name=None, link_prefix=None): deserializer = oe.spdx30.JSONLDDeserializer() - objset = ObjectSet(d) + objset = ObjectSet(d, name=name, link_prefix=link_prefix) try: with path.open("rb") as f: deserializer.read(f, objset) @@ -918,9 +925,9 @@ def jsonld_hash_path(_id): return Path("by-spdxid-hash") / h[:2], h -def load_jsonld_by_arch(d, arch, subdir, name, *, required=False): +def load_jsonld_by_arch(d, arch, subdir, name, *, required=False, link_prefix=None): path = jsonld_arch_path(d, arch, subdir, name) - objset = load_jsonld(d, path, required=required) + objset = load_jsonld(d, path, required=required, name=name, link_prefix=link_prefix) if objset is not None: return (objset, path) return (None, None) @@ -1049,8 +1056,8 @@ def find_root_obj_in_jsonld(d, subdir, fn_name, obj_type, **attr_filter): return spdx_obj, objset -def load_obj_in_jsonld(d, arch, subdir, fn_name, obj_type, **attr_filter): - objset, fn = load_jsonld_by_arch(d, arch, subdir, fn_name, required=True) +def load_obj_in_jsonld(d, arch, subdir, fn_name, obj_type, link_prefix=None, **attr_filter): + objset, fn = load_jsonld_by_arch(d, arch, subdir, fn_name, required=True, link_prefix=link_prefix) spdx_obj = objset.find_filter(obj_type, **attr_filter) if not spdx_obj: diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 5aeed5cd6f..ef829fbbf1 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -461,7 +461,7 @@ def create_spdx(d): if not include_vex in ("none", "current", "all"): bb.fatal("SPDX_INCLUDE_VEX must be one of 'none', 'current', 'all'") - build_objset = oe.sbom30.ObjectSet.new_objset(d, d.getVar("PN")) + build_objset = oe.sbom30.ObjectSet.new_objset(d, d.getVar("PN"), link_prefix="recipe") build = build_objset.new_task_build("recipe", "recipe") build_objset.set_element_alias(build) @@ -574,7 +574,7 @@ def create_spdx(d): bb.debug(1, "Creating SPDX for package %s" % pkg_name) - pkg_objset = oe.sbom30.ObjectSet.new_objset(d, pkg_name) + pkg_objset = oe.sbom30.ObjectSet.new_objset(d, pkg_name, link_prefix="package") spdx_package = pkg_objset.add_root( oe.spdx30.software_Package( @@ -793,7 +793,7 @@ def create_package_spdx(d): # Any element common to all packages that need to be referenced by ID # should be written into this objset set common_objset = oe.sbom30.ObjectSet.new_objset( - d, "%s-package-common" % d.getVar("PN") + d, "%s-package-common" % d.getVar("PN"), link_prefix="package" ) pkgdest = Path(d.getVar("PKGDEST")) @@ -812,6 +812,7 @@ def create_package_spdx(d): "packages-staging", pkg_name, oe.spdx30.software_Package, + link_prefix="package", software_primaryPurpose=oe.spdx30.software_SoftwarePurpose.install, ) @@ -1002,7 +1003,7 @@ def create_rootfs_spdx(d): with root_packages_file.open("r") as f: packages = json.load(f) - objset = oe.sbom30.ObjectSet.new_objset(d, "%s-%s" % (image_basename, machine)) + objset = oe.sbom30.ObjectSet.new_objset(d, "%s-%s" % (image_basename, machine), link_prefix="rootfs") rootfs = objset.add_root( oe.spdx30.software_Package( @@ -1037,7 +1038,7 @@ def create_image_spdx(d): image_basename = d.getVar("IMAGE_BASENAME") machine = d.getVar("MACHINE") - objset = oe.sbom30.ObjectSet.new_objset(d, "%s-%s" % (image_basename, machine)) + objset = oe.sbom30.ObjectSet.new_objset(d, "%s-%s" % (image_basename, machine), link_prefix="image") with manifest_path.open("r") as f: manifest = json.load(f) @@ -1150,7 +1151,7 @@ def sdk_create_spdx(d, sdk_type, spdx_work_dir, toolchain_outputname): sdk_name = toolchain_outputname + "-" + sdk_type sdk_packages = oe.sdk.sdk_list_installed_packages(d, sdk_type == "target") - objset = oe.sbom30.ObjectSet.new_objset(d, sdk_name) + objset = oe.sbom30.ObjectSet.new_objset(d, sdk_name, link_prefix="sdk") sdk_rootfs = objset.add_root( oe.spdx30.software_Package( From patchwork Mon Nov 25 08:14:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 53091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28BDED3B7E7 for ; Mon, 25 Nov 2024 08:15:12 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web10.14465.1732522504435309229 for ; Mon, 25 Nov 2024 00:15:04 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=1059dd1ea8=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4AP76maQ031692; Mon, 25 Nov 2024 08:15:03 GMT Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [147.11.82.252]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4336189qve-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Mon, 25 Nov 2024 08:15:03 +0000 (GMT) Received: from ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.43; Mon, 25 Nov 2024 00:15:00 -0800 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server id 15.1.2507.43 via Frontend Transport; Mon, 25 Nov 2024 00:15:00 -0800 From: Hongxu Jia To: , Subject: [oe-core][PATCH V3 2/3] sbom30.py: reduce redundant spdxid symlinks to save inode on host Date: Mon, 25 Nov 2024 00:14:59 -0800 Message-ID: <20241125081500.4048680-3-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241125081500.4048680-1-hongxu.jia@windriver.com> References: <20241125081500.4048680-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: I4en1yQsLWnbNCp9wHxnyJuYwtbpmQcx X-Proofpoint-GUID: I4en1yQsLWnbNCp9wHxnyJuYwtbpmQcx X-Authority-Analysis: v=2.4 cv=O65rvw9W c=1 sm=1 tr=0 ts=67443207 cx=c_pps a=/ZJR302f846pc/tyiSlYyQ==:117 a=/ZJR302f846pc/tyiSlYyQ==:17 a=VlfZXiiP6vEA:10 a=24AZYWMyAAAA:8 a=t7CeM3EgAAAA:8 a=spbkX35iY5P8FYnuM4IA:9 a=bG88sKzkDEFeXWNnvthB:22 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-25_05,2024-11-21_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 clxscore=1015 mlxscore=0 bulkscore=0 mlxlogscore=999 spamscore=0 priorityscore=1501 lowpriorityscore=0 adultscore=0 malwarescore=0 impostorscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2411250069 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Nov 2024 08:15:12 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/207724 In order to support all in-scope SPDX data within a single JSON-LD file for SPDX 3.0.1, Yocto's SBOM: - In native/target/nativesdk recipe, created spdxid-hash symlink for each element to point to the JSON-LD file that contains element details; - In image recipe, use spdxid-hash symlink to collect element details from varies of JSON-LD files While SPDX_INCLUDE_SOURCES = "1", it adds sources to JSON-LD file and create 2N+ spdxid-hash symlinks for N source files. (N for software_File, N for hasDeclaredLicense's Relationship) For large numbers of source files, adding an extra symlink -> real file will occupy one more inode (per file), which will need a slot in the OS's inode cache. In this situation, disk performance is slow and inode is used up quickly After commit [sbom30/spdx30: add link prefix and name to namespace of spdxId and alias] applied, the namespace of spdxId and alias in recipe and package jsonld differs. Use it to create symlink to jsonld, take recipe shadow, package shadow and package shadow-src for example: For recipe jsonld tmp/deploy/spdx/3.0.1/core2-64/recipes/shadow.spdx.json spdxId: http://spdx.org/spdxdocs/recipe-shadow-xxx/... alias: http://spdx.org/spdxdocs/openembedded-alias/recipe-shadow/UNIHASH/... link-name: recipe-shadow symlink: tmp/deploy/spdx/3.0.1/core2-64/by-spdxid-link/recipe-shadow.spdx.json -> ../recipes/shadow.spdx.json For package jsonld tmp/deploy/spdx/3.0.1/core2-64/packages/shadow.spdx.json spdxId: http://spdx.org/spdxdocs/package-shadow-xxx/... alias: http://spdx.org/spdxdocs/openembedded-alias/package-shadow/UNIHASH/... link-name: package-shadow symlink: tmp/deploy/spdx/3.0.1/core2-64/by-spdxid-link/package-shadow.spdx.json -> ../packages/shadow.spdx.json In package jsonld tmp/deploy/spdx/3.0.1/core2-64/packages/shadow-src.spdx.json spdxId: http://spdx.org/spdxdocs/package-shadow-src-xxx/... alias: http://spdx.org/spdxdocs/openembedded-alias/package-shadow-src/UNIHASH/... link-name: package-shadow-src symlink: tmp/deploy/spdx/3.0.1/core2-64/by-spdxid-link/package-shadow-src.spdx.json -> ../packages/shadow-src.spdx.json Build core-image-minimal with/without this commit, comparing the spdxid-link number, 7 281 824 -> 6 043 echo 'SPDX_INCLUDE_SOURCES = "1"' >> local.conf Without this commit: $ time bitbake core-image-minimal real 100m17.769s user 0m24.516s sys 0m4.334s $ find tmp/deploy/spdx/3.0.1/*/by-spdxid-hash -name "*.json" |wc -l 7281824 With this commit: $ time bitbake core-image-minimal real 85m12.994s user 0m20.423s sys 0m4.228s $ find tmp/deploy/spdx/3.0.1/*/by-spdxid-link -name "*.json" |wc -l 6043 Signed-off-by: Hongxu Jia --- meta/lib/oe/sbom30.py | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py index 28d251a7ac..014049e56d 100644 --- a/meta/lib/oe/sbom30.py +++ b/meta/lib/oe/sbom30.py @@ -919,10 +919,23 @@ def jsonld_arch_path(d, arch, subdir, name, deploydir=None): return deploydir / arch / subdir / (name + ".spdx.json") -def jsonld_hash_path(_id): - h = hashlib.sha256(_id.encode("utf-8")).hexdigest() +def jsonld_link_path(_id, d): + spdx_namespace_prefix = d.getVar("SPDX_NAMESPACE_PREFIX") + m = re.match(f"^{spdx_namespace_prefix}/openembedded-alias/([^/]+)/UNIHASH/", _id) + if m: + # Parse alias + # http://spdx.org/spdxdocs/openembedded-alias/recipe-shadow/UNIHASH/license/3_24_0/BSD-3-Clause -> recipe-shadow + link_path = m.group(1) + else: + m = re.match(f"^{spdx_namespace_prefix}/([^/]+)/", _id) + if m: + # Parse spdxId + # http://spdx.org/spdxdocs/recipe-shadow-10e66933-65cf-5a2d-9a1d-99b12a405441/55a7286167e0c1a871d49da1af6070709d52370a5b52fdea03d248452f919aaa/source/4 -> recipe-shadow + link_path = m.group(1)[0:-len(str(uuid.NAMESPACE_DNS))-1] + else: + bb.fatal("Invalid id %s, neither SPDX ID or alias" % _id) - return Path("by-spdxid-hash") / h[:2], h + return Path("by-spdxid-link"), link_path def load_jsonld_by_arch(d, arch, subdir, name, *, required=False, link_prefix=None): @@ -993,7 +1006,7 @@ def write_recipe_jsonld_doc( dest = jsonld_arch_path(d, pkg_arch, subdir, objset.doc.name, deploydir=deploydir) def link_id(_id): - hash_path = jsonld_hash_path(_id) + hash_path = jsonld_link_path(_id, d) link_name = jsonld_arch_path( d, @@ -1001,6 +1014,11 @@ def write_recipe_jsonld_doc( *hash_path, deploydir=deploydir, ) + + # Return if expected symlink exists + if link_name.is_symlink() and link_name.resolve() == dest: + return hash_path[-1] + try: link_name.parent.mkdir(exist_ok=True, parents=True) link_name.symlink_to(os.path.relpath(dest, link_name.parent)) @@ -1067,7 +1085,7 @@ def load_obj_in_jsonld(d, arch, subdir, fn_name, obj_type, link_prefix=None, **a def find_by_spdxid(d, spdxid, *, required=False): - return find_jsonld(d, *jsonld_hash_path(spdxid), required=required) + return find_jsonld(d, *jsonld_link_path(spdxid, d), required=required) def create_sbom(d, name, root_elements, add_objectsets=[]): From patchwork Mon Nov 25 08:15:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 53090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B243D3B7F0 for ; Mon, 25 Nov 2024 08:15:12 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web10.14466.1732522505054536611 for ; Mon, 25 Nov 2024 00:15:05 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=1059dd1ea8=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4AP76maR031692; Mon, 25 Nov 2024 08:15:03 GMT Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [147.11.82.252]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4336189qve-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Mon, 25 Nov 2024 08:15:03 +0000 (GMT) Received: from ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.43; Mon, 25 Nov 2024 00:15:00 -0800 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server id 15.1.2507.43 via Frontend Transport; Mon, 25 Nov 2024 00:15:00 -0800 From: Hongxu Jia To: , Subject: [oe-core][PATCH 3/3] oeqa/selftest: Add SPDX 3.0 include source cases for core_image_minimal build Date: Mon, 25 Nov 2024 00:15:00 -0800 Message-ID: <20241125081500.4048680-4-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241125081500.4048680-1-hongxu.jia@windriver.com> References: <20241125081500.4048680-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: bNYJk2hj1HBrvEzcsguIQ5VWI7Vik-N1 X-Proofpoint-GUID: bNYJk2hj1HBrvEzcsguIQ5VWI7Vik-N1 X-Authority-Analysis: v=2.4 cv=O65rvw9W c=1 sm=1 tr=0 ts=67443207 cx=c_pps a=/ZJR302f846pc/tyiSlYyQ==:117 a=/ZJR302f846pc/tyiSlYyQ==:17 a=VlfZXiiP6vEA:10 a=t7CeM3EgAAAA:8 a=X7M7THNEruu-k4-VEk8A:9 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-25_05,2024-11-21_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 clxscore=1015 mlxscore=0 bulkscore=0 mlxlogscore=999 spamscore=0 priorityscore=1501 lowpriorityscore=0 adultscore=0 malwarescore=0 impostorscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2411250069 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Nov 2024 08:15:12 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/207725 $ oe-selftest -r spdx.SPDX30Check.test_core_image_minimal_include_source 2024-11-09 09:17:54,600 - oe-selftest - INFO - Adding layer libraries: 2024-11-09 09:17:54,601 - oe-selftest - INFO - path-to/poky/meta/lib 2024-11-09 09:17:54,601 - oe-selftest - INFO - path-to/poky/meta-yocto-bsp/lib 2024-11-09 09:17:54,601 - oe-selftest - INFO - path-to/poky/meta-selftest/lib 2024-11-09 09:17:54,601 - oe-selftest - INFO - path-to/meta-openembedded/meta-oe/lib 2024-11-09 09:17:54,602 - oe-selftest - INFO - Checking base configuration is valid/parsable 2024-11-09 09:17:56,653 - oe-selftest - INFO - Adding: "include selftest.inc" in path-to/build_spdx3-st/conf/local.conf 2024-11-09 09:17:56,653 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2024-11-09 09:17:56,653 - oe-selftest - INFO - test_core_image_minimal_include_source (spdx.SPDX30Check.test_core_image_minimal_include_source) 2024-11-09 10:41:16,654 - oe-selftest - INFO - Keepalive message 2024-11-09 11:37:53,091 - oe-selftest - INFO - ... ok 2024-11-09 11:55:18,638 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-11-09 11:55:18,638 - oe-selftest - INFO - Ran 1 test in 9442.187s 2024-11-09 11:55:18,638 - oe-selftest - INFO - OK 2024-11-09 11:55:35,453 - oe-selftest - INFO - RESULTS: 2024-11-09 11:55:35,453 - oe-selftest - INFO - RESULTS - spdx.SPDX30Check.test_core_image_minimal_include_source: PASSED (8396.65s) 2024-11-09 11:55:35,490 - oe-selftest - INFO - SUMMARY: 2024-11-09 11:55:35,490 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 9442.187s 2024-11-09 11:55:35,490 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) Signed-off-by: Hongxu Jia --- meta/lib/oeqa/selftest/cases/spdx.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/spdx.py b/meta/lib/oeqa/selftest/cases/spdx.py index 8384070219..c785f5445f 100644 --- a/meta/lib/oeqa/selftest/cases/spdx.py +++ b/meta/lib/oeqa/selftest/cases/spdx.py @@ -174,6 +174,20 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): # Document should be fully linked self.check_objset_missing_ids(objset) + def test_core_image_minimal_include_source(self): + objset = self.check_recipe_spdx( + "core-image-minimal", + "{DEPLOY_DIR_IMAGE}/core-image-minimal-{MACHINE}.rootfs.spdx.json", + extraconf=textwrap.dedent( + """\ + SPDX_INCLUDE_SOURCES = "1" + """ + ), + ) + + # Document should be fully linked + self.check_objset_missing_ids(objset) + def test_core_image_minimal_sdk(self): objset = self.check_recipe_spdx( "core-image-minimal",