[bitbake-devel] data_smart: fix library name contains special word cause do_package failed

From: Xiangyu Chen <xiangyu.chen@windriver.com>

When the package library name contains like _append/_prepend words
and the pkg-config added those libraries to requires, the do_package
would report an error message:

"Variable xxxx contains an operation using the old override syntax.
Please convert this layer/metadata before attempting to use with a
newer bitbake"

This was found on latest version abseil-cpp package due to there is
a library name called "absl_log_internal_append_truncated"

So discard to check a variable with "Requires" header in setVar()

Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
---
 lib/bb/data_smart.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

On Wed, 2023-03-08 at 17:55 +0800, Xiangyu Chen wrote:
> From: Xiangyu Chen <xiangyu.chen@windriver.com>
> 
> When the package library name contains like _append/_prepend words
> and the pkg-config added those libraries to requires, the do_package
> would report an error message:
> 
> "Variable xxxx contains an operation using the old override syntax.
> Please convert this layer/metadata before attempting to use with a
> newer bitbake"
> 
> This was found on latest version abseil-cpp package due to there is
> a library name called "absl_log_internal_append_truncated"
> 
> So discard to check a variable with "Requires" header in setVar()
> 
> Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
> ---
>  lib/bb/data_smart.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> index c597dbad..fc33d34c 100644
> --- a/lib/bb/data_smart.py
> +++ b/lib/bb/data_smart.py
> @@ -539,7 +539,7 @@ class DataSmart(MutableMapping):
>      def setVar(self, var, value, **loginfo):
>          #print("var=" + str(var) + "  val=" + str(value))
>  
> -        if not var.startswith("__anon_") and ("_append" in var or "_prepend" in var or "_remove" in var):
> +        if not var.startswith("Requires") and not var.startswith("__anon_") and ("_append" in var or "_prepend" in var or "_remove" in var):
>              info = "%s" % var
>              if "file" in loginfo:
>                  info += " file: %s" % loginfo["file"]

"Requires" seems like a really strange name for a variable to start
with, where is that coming from? I don't think we should be hardcoding
OE-Core specific datastore usage inside bitbake. We're definitely not
merging that as is.

Is this from the pkgconfig code in do_package? If so we should probably
take a closer look at what that is doing as it doesn't sound good.

Cheers,

Richard

Hi Richard,

Thank you for your prompt response.


On 3/8/23 18:49, Richard Purdie wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Wed, 2023-03-08 at 17:55 +0800, Xiangyu Chen wrote:
>> From: Xiangyu Chen <xiangyu.chen@windriver.com>
>>
>> When the package library name contains like _append/_prepend words
>> and the pkg-config added those libraries to requires, the do_package
>> would report an error message:
>>
>> "Variable xxxx contains an operation using the old override syntax.
>> Please convert this layer/metadata before attempting to use with a
>> newer bitbake"
>>
>> This was found on latest version abseil-cpp package due to there is
>> a library name called "absl_log_internal_append_truncated"
>>
>> So discard to check a variable with "Requires" header in setVar()
>>
>> Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
>> ---
>>   lib/bb/data_smart.py | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
>> index c597dbad..fc33d34c 100644
>> --- a/lib/bb/data_smart.py
>> +++ b/lib/bb/data_smart.py
>> @@ -539,7 +539,7 @@ class DataSmart(MutableMapping):
>>       def setVar(self, var, value, **loginfo):
>>           #print("var=" + str(var) + "  val=" + str(value))
>>
>> -        if not var.startswith("__anon_") and ("_append" in var or "_prepend" in var or "_remove" in var):
>> +        if not var.startswith("Requires") and not var.startswith("__anon_") and ("_append" in var or "_prepend" in var or "_remove" in var):
>>               info = "%s" % var
>>               if "file" in loginfo:
>>                   info += " file: %s" % loginfo["file"]
> "Requires" seems like a really strange name for a variable to start
> with, where is that coming from? I don't think we should be hardcoding
> OE-Core specific datastore usage inside bitbake. We're definitely not
> merging that as is.
>
> Is this from the pkgconfig code in do_package? If so we should probably
> take a closer look at what that is doing as it doesn't sound good.

Yes, indeed, the word of "Requires" is coming from pkg-config .pc file[1].

When do_package read the .pc file, it would pass the "Requires" 
parameters to setVar in data_smart.py.

The setVar has a variable checking, when the variable contains 
"_append/_prepend/_remove", it would report an error and exit.

Coincidentally,the abseil-cpp has a library named 
"absl_log_internal_append_truncated" and declared at "Requires" section 
in a .pc file, it would hit the variables checking in setVar and this 
cause the latest version abseil-cpp cannot compile successfully. The 
packages(e.g. latest version protobuf) which depends latest version 
abseil-cpp also cannot update.

[1] https://people.freedesktop.org/~dbn/pkg-config-guide.html


Br,

Xiangyu

> Cheers,
>
> Richard
>

On Thu, 2023-03-09 at 08:46 +0800, Xiangyu Chen wrote:
> Hi Richard,
> 
> Thank you for your prompt response.
> 
> 
> On 3/8/23 18:49, Richard Purdie wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > 
> > On Wed, 2023-03-08 at 17:55 +0800, Xiangyu Chen wrote:
> > > From: Xiangyu Chen <xiangyu.chen@windriver.com>
> > > 
> > > When the package library name contains like _append/_prepend words
> > > and the pkg-config added those libraries to requires, the do_package
> > > would report an error message:
> > > 
> > > "Variable xxxx contains an operation using the old override syntax.
> > > Please convert this layer/metadata before attempting to use with a
> > > newer bitbake"
> > > 
> > > This was found on latest version abseil-cpp package due to there is
> > > a library name called "absl_log_internal_append_truncated"
> > > 
> > > So discard to check a variable with "Requires" header in setVar()
> > > 
> > > Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
> > > ---
> > >   lib/bb/data_smart.py | 2 +-
> > >   1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> > > index c597dbad..fc33d34c 100644
> > > --- a/lib/bb/data_smart.py
> > > +++ b/lib/bb/data_smart.py
> > > @@ -539,7 +539,7 @@ class DataSmart(MutableMapping):
> > >       def setVar(self, var, value, **loginfo):
> > >           #print("var=" + str(var) + "  val=" + str(value))
> > > 
> > > -        if not var.startswith("__anon_") and ("_append" in var or "_prepend" in var or "_remove" in var):
> > > +        if not var.startswith("Requires") and not var.startswith("__anon_") and ("_append" in var or "_prepend" in var or "_remove" in var):
> > >               info = "%s" % var
> > >               if "file" in loginfo:
> > >                   info += " file: %s" % loginfo["file"]
> > "Requires" seems like a really strange name for a variable to start
> > with, where is that coming from? I don't think we should be hardcoding
> > OE-Core specific datastore usage inside bitbake. We're definitely not
> > merging that as is.
> > 
> > Is this from the pkgconfig code in do_package? If so we should probably
> > take a closer look at what that is doing as it doesn't sound good.
> 
> Yes, indeed, the word of "Requires" is coming from pkg-config .pc file[1].
> 
> When do_package read the .pc file, it would pass the "Requires" 
> parameters to setVar in data_smart.py.
> 
> The setVar has a variable checking, when the variable contains 
> "_append/_prepend/_remove", it would report an error and exit.
> 
> Coincidentally,the abseil-cpp has a library named 
> "absl_log_internal_append_truncated" and declared at "Requires" section 
> in a .pc file, it would hit the variables checking in setVar and this 
> cause the latest version abseil-cpp cannot compile successfully. The 
> packages(e.g. latest version protobuf) which depends latest version 
> abseil-cpp also cannot update.
> 
> [1] https://people.freedesktop.org/~dbn/pkg-config-guide.html

I had a quick look at the pkgconfig code in do_package and I can't see
where this setVar is coming from. Can you point to the code which is
setting this?

As I said, we need to look at why this is happening. I'm very reluctant
to add special cases to generic code.

Cheers,

Richard