[2/3] spdx: add support for WITH operator

Adds the + operator for use in LICENSE field.
It is replaced by SPDX WITH operator.

Signed-off-by: Denis Osterland-Heim <denis.osterland@diehl.com>
---
 meta/lib/oe/license.py      | 4 ++--
 meta/lib/oe/spdx30_tasks.py | 4 ++++
 2 files changed, 6 insertions(+), 2 deletions(-)

     spdx_license_expression = " ".join(convert(l) for l in lic_split)
--
2.45.2

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/2-3-spdx-add-support-for-WITH-operator.patch

FAIL: test mbox format: Series has malformed diff lines. Create the series again using git-format-patch and ensure it applies using git am (test_mbox.TestMbox.test_mbox_format)

PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: Python-unidiff parse error (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: Parse error Hunk diff line expected: license_data):
SKIP: test Signed-off-by presence: Parse error Hunk diff line expected: license_data):
SKIP: test Upstream-Status presence: Parse error Hunk diff line expected: license_data):
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: Python-unidiff parse error (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Denis OSTERLAND-HEIM via lists.openembedded.org
> Sent: den 25 november 2024 12:57
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH 2/3] spdx: add support for WITH operator
> 
> Adds the + operator for use in LICENSE field.
> It is replaced by SPDX WITH operator.
> 
> Signed-off-by: Denis Osterland-Heim <denis.osterland@diehl.com>
> ---
>  meta/lib/oe/license.py      | 4 ++--
>  meta/lib/oe/spdx30_tasks.py | 4 ++++
>  2 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/lib/oe/license.py b/meta/lib/oe/license.py
> index 3c15db782f..223f98acd5 100644
> --- a/meta/lib/oe/license.py
> +++ b/meta/lib/oe/license.py
> @@ -47,9 +47,9 @@ class InvalidLicense(LicenseError):
>      def __str__(self):
>          return "invalid characters in license '%s'" % self.license
> 
> -license_operator_chars = '&|() '
> +license_operator_chars = '&|()+ '
>  license_operator = re.compile(r'([' + license_operator_chars + '])')
> -license_pattern = re.compile(r'[a-zA-Z0-9.+_\-]+$')
> +license_pattern = re.compile(r'[a-zA-Z0-9._\-]+$')
> 
>  class LicenseVisitor(ast.NodeVisitor):
>      """Get elements based on OpenEmbedded license strings"""
> diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
> index 5aeed5cd6f..17a1ce44e7 100644
> --- a/meta/lib/oe/spdx30_tasks.py
> +++ b/meta/lib/oe/spdx30_tasks.py
> @@ -97,6 +97,9 @@ def add_license_expression(d, objset, license_expression, license_data):
>          if l == "|":
>              return "OR"
> 
> +        if l == "+":
> +            return "WITH"
> +
>          if l == "CLOSED":
>              return "NONE"
> @@ -115,6 +118,7 @@ def add_license_expression(d, objset, license_expression, license_data):
>          .replace(")", " ) ")
>          .replace("|", " | ")
>          .replace("&", " & ")
> +        .replace("+", " + ")
>          .split()
>      )
>      spdx_license_expression = " ".join(convert(l) for l in lic_split)
> --
> 2.45.2

I have some concerns about this change:
* The above change does not really implement support for WITH. It 
  only supports converting a LICENSE expression containing a 
  WITH clause (expressed using the "+" operator) from bitbake 
  format into SPDX format. All the actual handling of WITH clauses 
  is missing.
* There are a lot of complexities related to the incompatible 
  license handling (which is the main reason for the license 
  handling in meta/lib/oe/license.py). There will be problems if 
  someone  specifies a LICENSE as "Foo + Exception | Bar". E.g., 
  consider what should happen if "Foo" is in INCOMPATIBLE_LICENSE.
* We also used to use "+" to indicate "or later". This can be seen 
  in obsolete_license_list(). Weird things will probably happen if 
  an old LICENSE like "GPL2+ & LGPLv2+" is encountered.

It would probably be easier to just add a GPL-2.0-with-syscall-exception 
(as was initially suggested), but containing the full GPL-2.0 license as 
well as the exception (compare to, e.g., Apache-2.0-with-LLVM-exception.

I also note that out of the six licenses used in OE-Core/OpenEmbedded 
that use exceptions:

Apache-2.0-with-LLVM-exception
GPL-2.0-with-GCC-exception
GPL-2.0-with-OpenSSL-exception
GPL-3-with-bison-exception
GPL-3.0-with-GCC-exception
GPL-3.0-with-autoconf-exception

only Apache-2.0-with-LLVM-exception and GPL-2.0-with-OpenSSL-exception 
contain the actual license text. The other use an "insert <license> here" 
reference. gcc has covered it by using "GPL-3.0-with-GCC-exception & 
GPL-3.0-only", but that is not the case for most other recipes.

//Peter