From patchwork Tue Oct 29 09:17:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 51483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3A8CD2AB05 for ; Tue, 29 Oct 2024 09:17:30 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web11.14824.1730193446912947186 for ; Tue, 29 Oct 2024 02:17:27 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=103298b06a=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49T4cLXm031504; Tue, 29 Oct 2024 09:17:25 GMT Received: from ala-exchng02.corp.ad.wrs.com (ala-exchng02.wrs.com [147.11.82.254]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 42gqd8kau1-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 29 Oct 2024 09:17:25 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 29 Oct 2024 02:17:23 -0700 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Tue, 29 Oct 2024 02:17:23 -0700 From: Hongxu Jia To: CC: Subject: [oe-core][PATCH V2 1/7] meta/lib/oe/sbom30.py: correct python list comprehension Date: Tue, 29 Oct 2024 02:17:17 -0700 Message-ID: <20241029091723.2851061-2-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241029091723.2851061-1-hongxu.jia@windriver.com> References: <20241029091723.2851061-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: qAGH4A77Hn4VTCxp3KqgnRbLT1WSchy2 X-Authority-Analysis: v=2.4 cv=dKj0m/Zb c=1 sm=1 tr=0 ts=6720a825 cx=c_pps a=K4BcnWQioVPsTJd46EJO2w==:117 a=K4BcnWQioVPsTJd46EJO2w==:17 a=DAUX931o1VcA:10 a=24AZYWMyAAAA:8 a=t7CeM3EgAAAA:8 a=kB6j27Rbq3LGHQ-lY6oA:9 a=bG88sKzkDEFeXWNnvthB:22 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-GUID: qAGH4A77Hn4VTCxp3KqgnRbLT1WSchy2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-29_04,2024-10-28_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxlogscore=859 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 clxscore=1015 spamscore=0 mlxscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2410290072 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 29 Oct 2024 09:17:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/206478 The python list comprehension is not right for list: >>> license_text_map = {'LicenseRef-FSF-Unlimited': 'http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/8d31e22acc4a8979f24dc24042692fb548fc8fc8d85d775ddac406abb122ceea/license-text/FSF-Unlimited'} >>> license_text = ((k, license_text_map[k]) for k in sorted(license_text_map.keys())) >>> print(license_text) at 0x7f8575173270> >>> [(k,v) for k, v in license_text] [] Change the () to [] to make it a list instead of a generator expression. Signed-off-by: Hongxu Jia Reviewed-by: Joshua Watt --- meta/lib/oe/sbom30.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py index 27ab5e45ac..8db90f30fd 100644 --- a/meta/lib/oe/sbom30.py +++ b/meta/lib/oe/sbom30.py @@ -577,9 +577,9 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): re.sub(r"[^a-zA-Z0-9_-]", "_", license_expression), ] - license_text = ( + license_text = [ (k, license_text_map[k]) for k in sorted(license_text_map.keys()) - ) + ] if not license_text: lic = self.find_filter( From patchwork Tue Oct 29 09:17:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 51484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3B80D2AB0A for ; Tue, 29 Oct 2024 09:17:30 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web10.14982.1730193447386613609 for ; Tue, 29 Oct 2024 02:17:27 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=103298b06a=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49T4cLXn031504; Tue, 29 Oct 2024 09:17:26 GMT Received: from ala-exchng02.corp.ad.wrs.com (ala-exchng02.wrs.com [147.11.82.254]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 42gqd8kau1-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 29 Oct 2024 09:17:26 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 29 Oct 2024 02:17:23 -0700 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Tue, 29 Oct 2024 02:17:23 -0700 From: Hongxu Jia To: CC: Subject: [oe-core][PATCH 2/7] meta/lib/oe/sbom30.py: correct typo Date: Tue, 29 Oct 2024 02:17:18 -0700 Message-ID: <20241029091723.2851061-3-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241029091723.2851061-1-hongxu.jia@windriver.com> References: <20241029091723.2851061-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: s0yi4XsA-w7LtBCvsEpdslRv5ILBoZIW X-Authority-Analysis: v=2.4 cv=dKj0m/Zb c=1 sm=1 tr=0 ts=6720a826 cx=c_pps a=K4BcnWQioVPsTJd46EJO2w==:117 a=K4BcnWQioVPsTJd46EJO2w==:17 a=DAUX931o1VcA:10 a=t7CeM3EgAAAA:8 a=uo5YQZFxS8OF35rn94wA:9 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-GUID: s0yi4XsA-w7LtBCvsEpdslRv5ILBoZIW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-29_04,2024-10-28_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxlogscore=894 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 clxscore=1015 spamscore=0 mlxscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2410290072 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 29 Oct 2024 09:17:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/206479 The isinstance expected 2 arguments Signed-off-by: Hongxu Jia Reviewed-by: Joshua Watt --- meta/lib/oe/sbom30.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py index 8db90f30fd..e3a9428668 100644 --- a/meta/lib/oe/sbom30.py +++ b/meta/lib/oe/sbom30.py @@ -631,7 +631,7 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): sha256_hash = bb.utils.sha256_file(path) for f in self.by_sha256_hash.get(sha256_hash, []): - if not isinstance(oe.spdx30.software_File): + if not isinstance(f, oe.spdx30.software_File): continue if purposes: From patchwork Tue Oct 29 09:17:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 51488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D419AD2AB0D for ; Tue, 29 Oct 2024 09:17:30 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web10.14984.1730193448165908799 for ; Tue, 29 Oct 2024 02:17:28 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=103298b06a=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49T4cLXo031504; Tue, 29 Oct 2024 09:17:27 GMT Received: from ala-exchng02.corp.ad.wrs.com (ala-exchng02.wrs.com [147.11.82.254]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 42gqd8kau1-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 29 Oct 2024 09:17:26 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 29 Oct 2024 02:17:23 -0700 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Tue, 29 Oct 2024 02:17:23 -0700 From: Hongxu Jia To: CC: Subject: [oe-core][PATCH V2 3/7] meta/lib/oe/sbom30.py: create hasDeclaredLicense relationship if file_licenses is empty Date: Tue, 29 Oct 2024 02:17:19 -0700 Message-ID: <20241029091723.2851061-4-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241029091723.2851061-1-hongxu.jia@windriver.com> References: <20241029091723.2851061-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: u90clAAj_j4CKBnKn7j_VS3klrN9m5bP X-Authority-Analysis: v=2.4 cv=dKj0m/Zb c=1 sm=1 tr=0 ts=6720a826 cx=c_pps a=K4BcnWQioVPsTJd46EJO2w==:117 a=K4BcnWQioVPsTJd46EJO2w==:17 a=DAUX931o1VcA:10 a=24AZYWMyAAAA:8 a=wECf3xPYAAAA:8 a=t7CeM3EgAAAA:8 a=fADrBXav0T-LPExS5kEA:9 a=bG88sKzkDEFeXWNnvthB:22 a=ccNonjl4-tybilS9-zgM:22 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-GUID: u90clAAj_j4CKBnKn7j_VS3klrN9m5bP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-29_04,2024-10-28_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxlogscore=999 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 clxscore=1015 spamscore=0 mlxscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2410290072 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 29 Oct 2024 09:17:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/206480 If file_licenses is empty, the hasDeclaredLicense relationship is from sourcefile to NoneElement. Such as { "type": "Relationship", ... "from": "http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb/sourcefile/3323", "relationshipType": "hasDeclaredLicense", "to": [ "NoneElement" ] }, According to Specification Version 3.0.1 NoneElement should be used if [1] the SPDX creator desires to assert that there are NO elements for the given context of use. NoAssertionElement should be used if [2] the SPDX creator has attempted to but cannot reach a reasonable objective determination; the SPDX creator has made no attempt to determine this field; or the SPDX creator has intentionally provided no information (no meaning should be implied by doing so). If we indicates to look for licenses and didn't find any. It should be NoAssertionElement other than NoneElement [1] https://spdx.github.io/spdx-spec/v3.0.1/model/Core/Individuals/NoneElement/ [2] https://spdx.github.io/spdx-spec/v3.0.1/model/Core/Individuals/NoAssertionElement/ Signed-off-by: Hongxu Jia --- meta/lib/oe/sbom30.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py index e3a9428668..7ae05c42a9 100644 --- a/meta/lib/oe/sbom30.py +++ b/meta/lib/oe/sbom30.py @@ -620,6 +620,11 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): for extracted_lic in oe.spdx_common.extract_licenses(filepath): file_licenses.add(self.new_license_expression(extracted_lic, license_data)) + # SPDX creator has attempted to but cannot reach a reasonable objective determination + # https://spdx.github.io/spdx-spec/v3.0.1/model/Core/Individuals/NoAssertionElement/ + if not file_licenses: + file_licenses = [oe.spdx30.Element.NoAssertionElement] + self.new_relationship( [spdx_file], oe.spdx30.RelationshipType.hasDeclaredLicense, From patchwork Tue Oct 29 09:17:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 51487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D73C5D2AB0E for ; Tue, 29 Oct 2024 09:17:30 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web10.14985.1730193448814315342 for ; Tue, 29 Oct 2024 02:17:29 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=103298b06a=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49T4cLXp031504; Tue, 29 Oct 2024 09:17:27 GMT Received: from ala-exchng02.corp.ad.wrs.com (ala-exchng02.wrs.com [147.11.82.254]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 42gqd8kau1-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 29 Oct 2024 09:17:27 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 29 Oct 2024 02:17:24 -0700 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Tue, 29 Oct 2024 02:17:24 -0700 From: Hongxu Jia To: CC: Subject: [oe-core][PATCH V2 4/7] meta/lib/oe/spdx30_tasks.py: improve debug log in add_package_files Date: Tue, 29 Oct 2024 02:17:20 -0700 Message-ID: <20241029091723.2851061-5-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241029091723.2851061-1-hongxu.jia@windriver.com> References: <20241029091723.2851061-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: NhFzcMZV-0KtIznUiD3v2kn16LZsDu1O X-Authority-Analysis: v=2.4 cv=dKj0m/Zb c=1 sm=1 tr=0 ts=6720a827 cx=c_pps a=K4BcnWQioVPsTJd46EJO2w==:117 a=K4BcnWQioVPsTJd46EJO2w==:17 a=DAUX931o1VcA:10 a=24AZYWMyAAAA:8 a=iGHA9ds3AAAA:8 a=t7CeM3EgAAAA:8 a=lI5sXhMdRfx_0E2x5mYA:9 a=aAh4nC1LCUkA:10 a=bG88sKzkDEFeXWNnvthB:22 a=nM-MV4yxpKKO9kiQg6Ot:22 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-GUID: NhFzcMZV-0KtIznUiD3v2kn16LZsDu1O X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-29_04,2024-10-28_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxlogscore=999 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 clxscore=1015 spamscore=0 mlxscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2410290072 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 29 Oct 2024 09:17:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/206481 While SPDX_INCLUDE_SOURCES = "1" [1], there are mess of `Adding file' in debug log ''' DEBUG: Adding file tmp/work/x86_64-linux/gettext-minimal-native/0.22.5/spdx/3.0.1/work/sources-unpack/COPYING to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/e2c2366654a818397af8b8ddb45fda88c2c71aa2d71695861f82376a658d8e66/document/gettext-minimal-native DEBUG: Adding file tmp/work/x86_64-linux/gettext-minimal-native/0.22.5/spdx/3.0.1/work/gettext-0.22.5/.tarball-version to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/e2c2366654a818397af8b8ddb45fda88c2c71aa2d71695861f82376a658d8e66/document/gettext-minimal-native ''' Summary the total number other than print for each file. ''' DEBUG: Added 7201 files to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/f5e0e04913ac4c595be791fc001d545a77519ed6ee8c743deef721ca0898bc94/document/gettext-minimal-native ''' [1] https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-SPDX_INCLUDE_SOURCES Signed-off-by: Hongxu Jia Reviewed-by: Joshua Watt --- meta/lib/oe/spdx30_tasks.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index e0b656d81f..5aeed5cd6f 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -155,8 +155,6 @@ def add_package_files( if filepath.is_symlink() or not filepath.is_file(): continue - bb.debug(1, "Adding file %s to %s" % (filepath, objset.doc._id)) - filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath) @@ -187,6 +185,8 @@ def add_package_files( file_counter += 1 + bb.debug(1, "Added %d files to %s" % (len(spdx_files), objset.doc._id)) + return spdx_files From patchwork Tue Oct 29 09:17:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 51485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2013D2AB01 for ; Tue, 29 Oct 2024 09:17:30 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web10.14986.1730193449553403419 for ; Tue, 29 Oct 2024 02:17:29 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=103298b06a=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49T4cLXq031504; Tue, 29 Oct 2024 09:17:28 GMT Received: from ala-exchng02.corp.ad.wrs.com (ala-exchng02.wrs.com [147.11.82.254]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 42gqd8kau1-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 29 Oct 2024 09:17:28 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 29 Oct 2024 02:17:24 -0700 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Tue, 29 Oct 2024 02:17:24 -0700 From: Hongxu Jia To: CC: Subject: [oe-core][PATCH 5/7] create-spdx-{2.2,3.0}: fix do_create_spdx Date: Tue, 29 Oct 2024 02:17:21 -0700 Message-ID: <20241029091723.2851061-6-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241029091723.2851061-1-hongxu.jia@windriver.com> References: <20241029091723.2851061-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 6r2_mKapRVXCZ_ZS1QFVYPPdtLvDUAdQ X-Authority-Analysis: v=2.4 cv=dKj0m/Zb c=1 sm=1 tr=0 ts=6720a828 cx=c_pps a=K4BcnWQioVPsTJd46EJO2w==:117 a=K4BcnWQioVPsTJd46EJO2w==:17 a=DAUX931o1VcA:10 a=t7CeM3EgAAAA:8 a=_jWdJo5MRqURDcVo7_AA:9 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-GUID: 6r2_mKapRVXCZ_ZS1QFVYPPdtLvDUAdQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-29_04,2024-10-28_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxlogscore=910 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 clxscore=1015 spamscore=0 mlxscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2410290072 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 29 Oct 2024 09:17:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/206482 Call function ${@create_spdx_source_deps(d)} or ${create_spdx_source_deps(d)} along with addtask not working, use task do_create_spdx flag 'depends' to instead Move create_spdx_source_deps to meta/lib/oe/spdx_common.py for both of SPDX 2.2 and SPDX 3.0 Signed-off-by: Hongxu Jia --- meta/classes/create-spdx-2.2.bbclass | 5 ++++- meta/classes/create-spdx-3.0.bbclass | 19 ++++--------------- meta/classes/spdx-common.bbclass | 21 +++++++++++++++++++++ meta/lib/oe/spdx_common.py | 4 ++++ 4 files changed, 33 insertions(+), 16 deletions(-) diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index cd1d6819bf..27242ecf70 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -584,7 +584,10 @@ addtask do_create_spdx_setscene do_create_spdx[dirs] = "${SPDXWORK}" do_create_spdx[cleandirs] = "${SPDXDEPLOY} ${SPDXWORK}" -do_create_spdx[depends] += "${PATCHDEPENDENCY}" +do_create_spdx[depends] += " \ + ${PATCHDEPENDENCY} \ + ${@create_spdx_source_deps(d)} \ +" python do_create_runtime_spdx() { from datetime import datetime, timezone diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass index 5f0590198f..bc23d2d211 100644 --- a/meta/classes/create-spdx-3.0.bbclass +++ b/meta/classes/create-spdx-3.0.bbclass @@ -132,22 +132,8 @@ addtask do_create_spdx after \ do_collect_spdx_deps \ do_deploy_source_date_epoch \ do_populate_sysroot do_package do_packagedata \ - ${create_spdx_source_deps(d)} \ before do_populate_sdk do_populate_sdk_ext do_build do_rm_work -def create_spdx_source_deps(d): - deps = [] - if d.getVar("SPDX_INCLUDE_SOURCES") == "1": - deps.extend([ - # do_unpack is a hack for now; we only need it to get the - # dependencies do_unpack already has so we can extract the source - # ourselves - "do_unpack", - # For kernel source code - "do_shared_workdir", - ]) - return " ".join(deps) - SSTATETASKS += "do_create_spdx" do_create_spdx[sstate-inputdirs] = "${SPDXDEPLOY}" do_create_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}" @@ -159,7 +145,10 @@ addtask do_create_spdx_setscene do_create_spdx[dirs] = "${SPDXWORK}" do_create_spdx[cleandirs] = "${SPDXDEPLOY} ${SPDXWORK}" -do_create_spdx[depends] += "${PATCHDEPENDENCY}" +do_create_spdx[depends] += " \ + ${PATCHDEPENDENCY} \ + ${@create_spdx_source_deps(d)} \ +" python do_create_package_spdx() { import oe.spdx30_tasks diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass index cd9cc0db98..ad02da5cd6 100644 --- a/meta/classes/spdx-common.bbclass +++ b/meta/classes/spdx-common.bbclass @@ -39,6 +39,27 @@ SPDX_CUSTOM_ANNOTATION_VARS ??= "" SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" +def create_spdx_source_deps(d): + import oe.spdx_common + + deps = [] + if d.getVar("SPDX_INCLUDE_SOURCES") == "1": + pn = d.getVar('PN') + # do_unpack is a hack for now; we only need it to get the + # dependencies do_unpack already has so we can extract the source + # ourselves + if oe.spdx_common.has_task(d, "do_unpack"): + deps.append("%s:do_unpack" % pn) + + if oe.spdx_common.is_work_shared_spdx(d) and \ + oe.spdx_common.process_sources(d): + # For kernel source code + if oe.spdx_common.has_task(d, "do_shared_workdir"): + deps.append("%s:do_shared_workdir" % pn) + + return " ".join(deps) + + python do_collect_spdx_deps() { # This task calculates the build time dependencies of the recipe, and is # required because while a task can deptask on itself, those dependencies diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 1ea55419ae..7a85579f64 100644 --- a/meta/lib/oe/spdx_common.py +++ b/meta/lib/oe/spdx_common.py @@ -226,6 +226,10 @@ def get_patched_src(d): d.setVar("WORKDIR", workdir) +def has_task(d, task): + return bool(d.getVarFlag(task, "task", False)) and not bool(d.getVarFlag(task, "noexec", False)) + + def fetch_data_to_uri(fd, name): """ Translates a bitbake FetchData to a string URI From patchwork Tue Oct 29 09:17:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 51486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B71ABD2AB08 for ; Tue, 29 Oct 2024 09:17:30 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web11.14826.1730193450163705881 for ; Tue, 29 Oct 2024 02:17:30 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=103298b06a=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49T4cLXr031504; Tue, 29 Oct 2024 09:17:29 GMT Received: from ala-exchng02.corp.ad.wrs.com (ala-exchng02.wrs.com [147.11.82.254]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 42gqd8kau1-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 29 Oct 2024 09:17:28 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 29 Oct 2024 02:17:24 -0700 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Tue, 29 Oct 2024 02:17:24 -0700 From: Hongxu Jia To: CC: Subject: [oe-core][PATCH V2 6/7] create-spdx-{2.2,3.0}: support SPDX include source for work-share directory Date: Tue, 29 Oct 2024 02:17:22 -0700 Message-ID: <20241029091723.2851061-7-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241029091723.2851061-1-hongxu.jia@windriver.com> References: <20241029091723.2851061-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 7ZLsfA7ga3N8M6zjRImZmvwc32tGh8Ed X-Authority-Analysis: v=2.4 cv=dKj0m/Zb c=1 sm=1 tr=0 ts=6720a828 cx=c_pps a=K4BcnWQioVPsTJd46EJO2w==:117 a=K4BcnWQioVPsTJd46EJO2w==:17 a=DAUX931o1VcA:10 a=iGHA9ds3AAAA:8 a=t7CeM3EgAAAA:8 a=HEwNEoMtOsuOwCeMpPQA:9 a=aAh4nC1LCUkA:10 a=nM-MV4yxpKKO9kiQg6Ot:22 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-GUID: 7ZLsfA7ga3N8M6zjRImZmvwc32tGh8Ed X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-29_04,2024-10-28_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxlogscore=999 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 clxscore=1015 spamscore=0 mlxscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2410290072 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 29 Oct 2024 09:17:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/206483 Originally, while SPDX_INCLUDE_SOURCES = "1" [1], there is bug in scan for gcc, libgcc in which the sources locates in work-share directory. Copy source from ${WORKDIR} to ${SPDXWORK} did not satisfy the situation while ${S} was not included in ${WORKDIR} This commit aim to support SPDX include source for work-share directory 1. If is_work_shared_spdx, Copy source from ${S} to ${SPDXWORK}, normally the dest dir in ${SPDXWORK} has the same basename dir of ${S}; but for kernel source, rename basename dir 'kernel-source' to ${BP} (${BPN}-${PV}) 2. For SPDX source copy, do hard link copy to save copy time 3. Move do_patch to no work shared situation along with do_unpack 4. Tweak task do_create_spdx dependencies to assure the patched source in work share is ready for SPDX source copy 5. Remove bb.data.inherits_class('kernel', d) from is_work_shared_spdx, the kernel source locates in 'work-shared', test kernel.bbclass is not necessary [1] https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-SPDX_INCLUDE_SOURCES Signed-off-by: Hongxu Jia --- meta/classes/spdx-common.bbclass | 9 +++++++ meta/lib/oe/spdx_common.py | 41 ++++++++++++-------------------- 2 files changed, 24 insertions(+), 26 deletions(-) diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass index ad02da5cd6..d3cdc8b6ce 100644 --- a/meta/classes/spdx-common.bbclass +++ b/meta/classes/spdx-common.bbclass @@ -57,6 +57,15 @@ def create_spdx_source_deps(d): if oe.spdx_common.has_task(d, "do_shared_workdir"): deps.append("%s:do_shared_workdir" % pn) + # For gcc-source-${PV} source code + if oe.spdx_common.has_task(d, "do_preconfigure"): + deps.append("%s:do_preconfigure" % pn) + elif oe.spdx_common.has_task(d, "do_patch"): + deps.append("%s:do_patch" % pn) + # For gcc-cross-x86_64 source code + elif oe.spdx_common.has_task(d, "do_configure"): + deps.append("%s:do_configure" % pn) + return " ".join(deps) diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 7a85579f64..ff18d91780 100644 --- a/meta/lib/oe/spdx_common.py +++ b/meta/lib/oe/spdx_common.py @@ -38,7 +38,7 @@ def extract_licenses(filename): def is_work_shared_spdx(d): - return bb.data.inherits_class("kernel", d) or ("work-shared" in d.getVar("WORKDIR")) + return '/work-shared/' in d.getVar('S') def load_spdx_license_data(d): @@ -74,11 +74,6 @@ def process_sources(d): if d.getVar("PN") == "shadow-sysroot": return False - # We just archive gcc-source for all the gcc related recipes - if d.getVar("BPN") in ["gcc", "libgcc"]: - bb.debug(1, "spdx: There is bug in scan of %s is, do nothing" % pn) - return False - return True @@ -190,34 +185,28 @@ def get_patched_src(d): bb.utils.mkdirhier(d.getVar("B")) bb.build.exec_func("do_unpack", d) - # Copy source of kernel to spdx_workdir + + if d.getVar("SRC_URI") != "": + bb.build.exec_func("do_patch", d) + + # Copy source from work-share to spdx_workdir if is_work_shared_spdx(d): - share_src = d.getVar("WORKDIR") + share_src = d.getVar('S') d.setVar("WORKDIR", spdx_workdir) d.setVar("STAGING_DIR_NATIVE", spdx_sysroot_native) + # Copy source to ${SPDXWORK}, same basename dir of ${S}; src_dir = ( spdx_workdir + "/" - + d.getVar("PN") - + "-" - + d.getVar("PV") - + "-" - + d.getVar("PR") + + os.path.basename(share_src) ) - bb.utils.mkdirhier(src_dir) + # For kernel souce, rename suffix dir 'kernel-source' + # to ${BP} (${BPN}-${PV}) if bb.data.inherits_class("kernel", d): - share_src = d.getVar("STAGING_KERNEL_DIR") - cmd_copy_share = "cp -rf " + share_src + "/* " + src_dir + "/" - cmd_copy_shared_res = os.popen(cmd_copy_share).read() - bb.note("cmd_copy_shared_result = " + cmd_copy_shared_res) - - git_path = src_dir + "/.git" - if os.path.exists(git_path): - shutils.rmtree(git_path) - - # Make sure gcc and kernel sources are patched only once - if not (d.getVar("SRC_URI") == "" or is_work_shared_spdx(d)): - bb.build.exec_func("do_patch", d) + src_dir = spdx_workdir + "/" + d.getVar('BP') + + bb.note(f"copyhardlinktree {share_src} to {src_dir}") + oe.path.copyhardlinktree(share_src, src_dir) # Some userland has no source. if not os.path.exists(spdx_workdir): From patchwork Tue Oct 29 09:17:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 51489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD7B4D2AB06 for ; Tue, 29 Oct 2024 09:17:40 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web10.14987.1730193450719766373 for ; Tue, 29 Oct 2024 02:17:30 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=103298b06a=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49T4cLXs031504; Tue, 29 Oct 2024 09:17:29 GMT Received: from ala-exchng02.corp.ad.wrs.com (ala-exchng02.wrs.com [147.11.82.254]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 42gqd8kau1-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 29 Oct 2024 09:17:29 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ALA-EXCHNG02.corp.ad.wrs.com (147.11.82.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 29 Oct 2024 02:17:24 -0700 Received: from ala-lpggp7.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Tue, 29 Oct 2024 02:17:24 -0700 From: Hongxu Jia To: CC: Subject: [oe-core][PATCH 7/7] oeqa/selftest: Add SPDX 3.0 include source case for work-share Date: Tue, 29 Oct 2024 02:17:23 -0700 Message-ID: <20241029091723.2851061-8-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241029091723.2851061-1-hongxu.jia@windriver.com> References: <20241029091723.2851061-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: qXLjSntZ5eGWxfjPbf0Y2YnRFqDOr_cF X-Authority-Analysis: v=2.4 cv=dKj0m/Zb c=1 sm=1 tr=0 ts=6720a829 cx=c_pps a=K4BcnWQioVPsTJd46EJO2w==:117 a=K4BcnWQioVPsTJd46EJO2w==:17 a=DAUX931o1VcA:10 a=24AZYWMyAAAA:8 a=t7CeM3EgAAAA:8 a=2V6DY6BgAAAA:8 a=YKlqnSHKsa4dvJv7hPcA:9 a=bG88sKzkDEFeXWNnvthB:22 a=FdTzh2GWekK77mhwV6Dw:22 a=ldqKKs2zR4t-S6fqr-1n:22 X-Proofpoint-GUID: qXLjSntZ5eGWxfjPbf0Y2YnRFqDOr_cF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-29_04,2024-10-28_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxlogscore=999 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 clxscore=1015 spamscore=0 mlxscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.21.0-2409260000 definitions=main-2410290072 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 29 Oct 2024 09:17:40 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/206484 Build gcc and check gcc-14.2.0/README in objset is available $ oe-selftest -r spdx.SPDX30Check.test_gcc_include_source ... 2024-10-26 01:24:57,063 - oe-selftest - INFO - test_gcc_include_source (spdx.SPDX30Check.test_gcc_include_source) 2024-10-26 01:28:24,204 - oe-selftest - INFO - The spdxId of gcc-14.2.0/README in gcc.spdx.json is http://spdx.org/spdxdocs/gcc-f2eaeb0d-b54b-53ba-899a-8c36c21139bf/88d5068ffd41e5ea6b4e0dd390b23bf499bb2b6674a41e09eaf2a887eced16c8/sourcefile/42 2024-10-26 01:28:26,369 - oe-selftest - INFO - ... ok 2024-10-26 01:28:33,315 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-10-26 01:28:33,316 - oe-selftest - INFO - Ran 1 test in 216.457s 2024-10-26 01:28:33,316 - oe-selftest - INFO - OK 2024-10-26 01:28:45,254 - oe-selftest - INFO - RESULTS: 2024-10-26 01:28:45,254 - oe-selftest - INFO - RESULTS - spdx.SPDX30Check.test_gcc_include_source: PASSED (209.31s) 2024-10-26 01:28:45,260 - oe-selftest - INFO - SUMMARY: 2024-10-26 01:28:45,260 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 216.457s 2024-10-26 01:28:45,260 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) Signed-off-by: Hongxu Jia --- meta/lib/oeqa/selftest/cases/spdx.py | 29 ++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/spdx.py b/meta/lib/oeqa/selftest/cases/spdx.py index be595babb3..8384070219 100644 --- a/meta/lib/oeqa/selftest/cases/spdx.py +++ b/meta/lib/oeqa/selftest/cases/spdx.py @@ -110,6 +110,7 @@ class SPDX3CheckBase(object): "SDKMACHINE", "SDK_DEPLOY", "SPDX_VERSION", + "SSTATE_PKGARCH", "TOOLCHAIN_OUTPUTNAME", ], target_name, @@ -136,6 +137,34 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): "{DEPLOY_DIR_SPDX}/{MACHINE_ARCH}/packages/base-files.spdx.json", ) + + def test_gcc_include_source(self): + import oe.spdx30 + + objset = self.check_recipe_spdx( + "gcc", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/recipes/gcc.spdx.json", + extraconf=textwrap.dedent( + """\ + SPDX_INCLUDE_SOURCES = "1" + """ + ), + ) + + gcc_pv = get_bb_var("PV", "gcc") + filename = f'gcc-{gcc_pv}/README' + found = False + for software_file in objset.foreach_type(oe.spdx30.software_File): + if software_file.name == filename: + found = True + self.logger.info(f"The spdxId of {filename} in gcc.spdx.json is {software_file.spdxId}") + break + + self.assertTrue( + found, + f"Not found source file {filename} in gcc.spdx.json\n" + ) + def test_core_image_minimal(self): objset = self.check_recipe_spdx( "core-image-minimal",