| Message ID | 20241212110932.2063844-1-hongxu.jia@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | meta/lib/oe/spdx30_tasks.py: fix alias in simplelicensing_customIdToUri not extracted | expand |
On Thu, Dec 12, 2024 at 4:09 AM Hongxu Jia <hongxu.jia@windriver.com> wrote: > > After commit [spdx 3.0: Rework how SPDX aliases are linked] applied, it set > license_text_map with alias other than actual spdxId > > The property of simplelicensing_customIdToUri is ListProp(ObjectProp(DictionaryEntry))), > and class DictionaryEntry has key and value, the property of value is StringProp other > than ObjectProp in which could not support to decode/extract alias with actual spdxId in > image jsonld file > > $ bitbake core-image-minimal > $ vim tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.spdx.json > ... > "simplelicensing_customIdToUri": [ > { > "type": "DictionaryEntry", > "key": "LicenseRef-PD", > "value": "http://spdxdocs.org/openembedded-alias/by-doc-hash/d53e90e23b12c4ad640809a74a810e86f31c76cdbdf36487712d22a33d53362a/sqlite3-native/UNIHASH/license-text/PD" > } > ], > ... > > After applying this commit, revert [spdx 3.0: Rework how SPDX aliases are linked] > to set actual spdxId other than alias to dictionary license_text_map > ... > "simplelicensing_customIdToUri": [ > { > "type": "DictionaryEntry", > "key": "LicenseRef-PD", > "value": "http://spdx.org/spdxdocs/sqlite3-native-e5cc0672-d8dd-57e8-a2df-fe4615831fda/162c62b5b011cd3f82f413b3dae4d6d1542201552c964d5ce69fe170e0720b85/license-text/PD" > } > ] > ... > > Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> > --- > meta/lib/oe/spdx30_tasks.py | 4 +--- > 1 file changed, 1 insertion(+), 3 deletions(-) > > diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py > index 036c58bf4b..82fcfe7a63 100644 > --- a/meta/lib/oe/spdx30_tasks.py > +++ b/meta/lib/oe/spdx30_tasks.py > @@ -107,9 +107,7 @@ def add_license_expression(d, objset, license_expression, license_data): > > spdx_license = "LicenseRef-" + l > if spdx_license not in license_text_map: > - license_text_map[spdx_license] = oe.sbom30.get_element_link_id( > - add_license_text(l) > - ) > + license_text_map[spdx_license] = add_license_text(l)._id Ah, sorry, this will not work because the SPDX IDs can be wrong anytime that documents are referenced across sstate objects. This is why we have aliases in the first place so that they can be "looked up" independently of their actual SPDX ID, until they are all linked together in the final SPDX document. This works for normal object references just fine because the alias SPDX IDs are "missing" when the document is merged, and our code knows how to pull in those SPDX IDs by the "missing" alias to resolve them. Unfortunately, since a DictionaryEntry holds a free form string for the value, that doesn't work here. Instead, we will need to add special code in the linking to manually go through all of the simplelicensing_customIdToUri DictionaryEntry items and resolve any aliases to actual objects. I think that needs to go in expand_collection() in sbom30.py > > return spdx_license > > -- > 2.25.1 >
Hi Joshua, I've submitted two solutions to fix the issue, One is make StringProp to decode SPDX alias with actual ID, which refers class ObjectProp's decode [1] Another is add special code in expand_collection as you suggested Would you please have a selection on them, or other better ideas/suggestions? [1] https://github.com/openembedded/openembedded-core/blob/master/meta/lib/oe/spdx30.py#L293 //Hongxu On 12/14/24 07:44, Joshua Watt wrote: > CAUTION: This email comes from a non Wind River email account! > Do not click links or open attachments unless you recognize the sender and know the content is safe. > > On Thu, Dec 12, 2024 at 4:09 AM Hongxu Jia<hongxu.jia@windriver.com> wrote: >> After commit [spdx 3.0: Rework how SPDX aliases are linked] applied, it set >> license_text_map with alias other than actual spdxId >> >> The property of simplelicensing_customIdToUri is ListProp(ObjectProp(DictionaryEntry))), >> and class DictionaryEntry has key and value, the property of value is StringProp other >> than ObjectProp in which could not support to decode/extract alias with actual spdxId in >> image jsonld file >> >> $ bitbake core-image-minimal >> $ vim tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.spdx.json >> ... >> "simplelicensing_customIdToUri": [ >> { >> "type": "DictionaryEntry", >> "key": "LicenseRef-PD", >> "value":"http://spdxdocs.org/openembedded-alias/by-doc-hash/d53e90e23b12c4ad640809a74a810e86f31c76cdbdf36487712d22a33d53362a/sqlite3-native/UNIHASH/license-text/PD" >> } >> ], >> ... >> >> After applying this commit, revert [spdx 3.0: Rework how SPDX aliases are linked] >> to set actual spdxId other than alias to dictionary license_text_map >> ... >> "simplelicensing_customIdToUri": [ >> { >> "type": "DictionaryEntry", >> "key": "LicenseRef-PD", >> "value":"http://spdx.org/spdxdocs/sqlite3-native-e5cc0672-d8dd-57e8-a2df-fe4615831fda/162c62b5b011cd3f82f413b3dae4d6d1542201552c964d5ce69fe170e0720b85/license-text/PD" >> } >> ] >> ... >> >> Signed-off-by: Hongxu Jia<hongxu.jia@windriver.com> >> --- >> meta/lib/oe/spdx30_tasks.py | 4 +--- >> 1 file changed, 1 insertion(+), 3 deletions(-) >> >> diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py >> index 036c58bf4b..82fcfe7a63 100644 >> --- a/meta/lib/oe/spdx30_tasks.py >> +++ b/meta/lib/oe/spdx30_tasks.py >> @@ -107,9 +107,7 @@ def add_license_expression(d, objset, license_expression, license_data): >> >> spdx_license = "LicenseRef-" + l >> if spdx_license not in license_text_map: >> - license_text_map[spdx_license] = oe.sbom30.get_element_link_id( >> - add_license_text(l) >> - ) >> + license_text_map[spdx_license] = add_license_text(l)._id > Ah, sorry, this will not work because the SPDX IDs can be wrong > anytime that documents are referenced across sstate objects. This is > why we have aliases in the first place so that they can be "looked up" > independently of their actual SPDX ID, until they are all linked > together in the final SPDX document. This works for normal object > references just fine because the alias SPDX IDs are "missing" when the > document is merged, and our code knows how to pull in those SPDX IDs > by the "missing" alias to resolve them. > > Unfortunately, since a DictionaryEntry holds a free form string for > the value, that doesn't work here. Instead, we will need to add > special code in the linking to manually go through all of the > simplelicensing_customIdToUri DictionaryEntry items and resolve any > aliases to actual objects. I think that needs to go in > expand_collection() in sbom30.py > >> return spdx_license >> >> -- >> 2.25.1 >>
diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 036c58bf4b..82fcfe7a63 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -107,9 +107,7 @@ def add_license_expression(d, objset, license_expression, license_data): spdx_license = "LicenseRef-" + l if spdx_license not in license_text_map: - license_text_map[spdx_license] = oe.sbom30.get_element_link_id( - add_license_text(l) - ) + license_text_map[spdx_license] = add_license_text(l)._id return spdx_license
After commit [spdx 3.0: Rework how SPDX aliases are linked] applied, it set license_text_map with alias other than actual spdxId The property of simplelicensing_customIdToUri is ListProp(ObjectProp(DictionaryEntry))), and class DictionaryEntry has key and value, the property of value is StringProp other than ObjectProp in which could not support to decode/extract alias with actual spdxId in image jsonld file $ bitbake core-image-minimal $ vim tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.spdx.json ... "simplelicensing_customIdToUri": [ { "type": "DictionaryEntry", "key": "LicenseRef-PD", "value": "http://spdxdocs.org/openembedded-alias/by-doc-hash/d53e90e23b12c4ad640809a74a810e86f31c76cdbdf36487712d22a33d53362a/sqlite3-native/UNIHASH/license-text/PD" } ], ... After applying this commit, revert [spdx 3.0: Rework how SPDX aliases are linked] to set actual spdxId other than alias to dictionary license_text_map ... "simplelicensing_customIdToUri": [ { "type": "DictionaryEntry", "key": "LicenseRef-PD", "value": "http://spdx.org/spdxdocs/sqlite3-native-e5cc0672-d8dd-57e8-a2df-fe4615831fda/162c62b5b011cd3f82f413b3dae4d6d1542201552c964d5ce69fe170e0720b85/license-text/PD" } ] ... Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> --- meta/lib/oe/spdx30_tasks.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)