[v5,1/2] binutils: Fix packaging of bfd-plugins

From: Harish Sadineni <Harish.Sadineni@windriver.com>

Fixes the follwing error while building nativesdk-binutils with i686-mingw32 as SDKMACHINE:
ERROR: nativesdk-binutils-2.43.1-r0 do_package: QA Issue: nativesdk-binutils: Files/directories
were installed but not shipped in any package:
   /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-w64-mingw32/usr/lib/bfd-plugins
   /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-w64-mingw32/usr/lib/bfd-plugins/libdep.dll

Signed-off-by: Harish Sadineni <Harish.Sadineni@windriver.com>
---
 meta/recipes-devtools/binutils/binutils_2.43.1.bb | 1 +
 1 file changed, 1 insertion(+)

On Thu, 2024-09-19 at 05:48 -0700, Sadineni, Harish via lists.openembedded.org wrote:
> From: Harish Sadineni <Harish.Sadineni@windriver.com>
> 
> Fixes the follwing error while building nativesdk-binutils with i686-mingw32 as SDKMACHINE:
> ERROR: nativesdk-binutils-2.43.1-r0 do_package: QA Issue: nativesdk-binutils: Files/directories
> were installed but not shipped in any package:
>    /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-w64-mingw32/usr/lib/bfd-plugins
>    /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-w64-mingw32/usr/lib/bfd-plugins/libdep.dll
> 
> Signed-off-by: Harish Sadineni <Harish.Sadineni@windriver.com>
> ---
>  meta/recipes-devtools/binutils/binutils_2.43.1.bb | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/recipes-devtools/binutils/binutils_2.43.1.bb b/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> index 2cce40f1ef..eb7c16cbc4 100644
> --- a/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> +++ b/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> @@ -72,5 +72,6 @@ SRC_URI:append:class-nativesdk =  " file://0003-binutils-nativesdk-Search-for-al
>  
>  USE_ALTERNATIVES_FOR:class-nativesdk = ""
>  FILES:${PN}:append:class-nativesdk = " ${bindir}"
> +FILES:${PN}:append:class-nativesdk = " ${libdir}/bfd-plugins"
>  
>  BBCLASSEXTEND = "native nativesdk"

I ended up digging into this a lot more. I was able to reproduce the
failure locally but that implies the autobuilder isn't testing this
combination. According to the configuration, it should be so I'm a bit
puzzled why the autobuilder doesn't see it.

The autobuilder config for this is here:

https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json#n1362

Resulting in a build such as this:

https://valkyrie.yoctoproject.org/#/builders/7/builds/162

I would like to understand why automated testing isn't catching this. 

Getting back to the patch, I noticed that bintutils.inc already does:

 FILES:${PN} = " \
        ${bindir}/${TARGET_PREFIX}* \
        ${libdir}/lib*.so.* \
        ${libdir}/bfd-plugins/lib*.so \
...

and the issue is that it uses .so and the recipe is generating a .dll
file.

I'd note that bitbake.conf has:

 SOLIBS = ".so.*"
 SOLIBS:darwin = ".dylib"
 
 SOLIBSDEV = ".so"
 # Due to the ordering of PACKAGES and the naming of the dev symlinks on darwin,
 # we can't make the symlinks end up in the -dev packages easily at this point. This hack
 # at least means builds aren't completely broken and symlinks don't take up much space.
 SOLIBSDEV:darwin = ".dylibbroken"

with the intent of being able to abstract out dylib/so/dll references.
I'm wondering if we should add:

SOLIBS:mingw32 = ".dll"
SOLIBSDEV:mingw32 = ".dll"

which would then let us change the binutils.inc entry to:

${libdir}/bfd-plugins/lib*${SOLIBSDEV}

In testing this I found it breaks zstd and zlib since those always
generate .so files even on mingw. I've hardcoded SOLIBS and SOLIBSDEV
in the bbappends in mingw to counteract that:

https://git.yoctoproject.org/meta-mingw/commit/?id=db1f53f73e95499a5f0b0edc45dd9cc9718d3c8a

I think these changes should combine to fix the underlying problem and
also make it easier to fix things like this in future.

They probably need wider testing though.

Cheers,

Richard

In general, we don't really support any nativesdk- recipes in MinGW;
it's pretty difficult to get recipes working for MinGW in general, so
we are pretty focused on just providing the cross-compiler utilities
so that the SDK can be used to compile for your target device, and
less so to provide utilities for running on Windows proper. A lot of
recipes are pretty much impossible to compile as nativesdk- (e.g.
Python).

In general, my recommendation would be to use the MinGW SDK to provide
the cross compiler, and if you want additional tools in your Windows
environment, use something like msys2 to provide them (or compile your
own versions). IMHO, this a much more maintainable long term solution,
since I don't think it's going to be particularly easy to allow
arbitrary recipes to be compiled as nativesdk- for MinGW, and
msys2/mingw has already solved this problem anyway.

On Thu, Sep 19, 2024 at 7:41 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2024-09-19 at 05:48 -0700, Sadineni, Harish via lists.openembedded.org wrote:
> > From: Harish Sadineni <Harish.Sadineni@windriver.com>
> >
> > Fixes the follwing error while building nativesdk-binutils with i686-mingw32 as SDKMACHINE:
> > ERROR: nativesdk-binutils-2.43.1-r0 do_package: QA Issue: nativesdk-binutils: Files/directories
> > were installed but not shipped in any package:
> >    /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-w64-mingw32/usr/lib/bfd-plugins
> >    /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-w64-mingw32/usr/lib/bfd-plugins/libdep.dll
> >
> > Signed-off-by: Harish Sadineni <Harish.Sadineni@windriver.com>
> > ---
> >  meta/recipes-devtools/binutils/binutils_2.43.1.bb | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/meta/recipes-devtools/binutils/binutils_2.43.1.bb b/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> > index 2cce40f1ef..eb7c16cbc4 100644
> > --- a/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> > +++ b/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> > @@ -72,5 +72,6 @@ SRC_URI:append:class-nativesdk =  " file://0003-binutils-nativesdk-Search-for-al
> >
> >  USE_ALTERNATIVES_FOR:class-nativesdk = ""
> >  FILES:${PN}:append:class-nativesdk = " ${bindir}"
> > +FILES:${PN}:append:class-nativesdk = " ${libdir}/bfd-plugins"
> >
> >  BBCLASSEXTEND = "native nativesdk"
>
> I ended up digging into this a lot more. I was able to reproduce the
> failure locally but that implies the autobuilder isn't testing this
> combination. According to the configuration, it should be so I'm a bit
> puzzled why the autobuilder doesn't see it.
>
> The autobuilder config for this is here:
>
> https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json#n1362
>
> Resulting in a build such as this:
>
> https://valkyrie.yoctoproject.org/#/builders/7/builds/162
>
> I would like to understand why automated testing isn't catching this.
>
> Getting back to the patch, I noticed that bintutils.inc already does:
>
>  FILES:${PN} = " \
>         ${bindir}/${TARGET_PREFIX}* \
>         ${libdir}/lib*.so.* \
>         ${libdir}/bfd-plugins/lib*.so \
> ...
>
> and the issue is that it uses .so and the recipe is generating a .dll
> file.
>
> I'd note that bitbake.conf has:
>
>  SOLIBS = ".so.*"
>  SOLIBS:darwin = ".dylib"
>
>  SOLIBSDEV = ".so"
>  # Due to the ordering of PACKAGES and the naming of the dev symlinks on darwin,
>  # we can't make the symlinks end up in the -dev packages easily at this point. This hack
>  # at least means builds aren't completely broken and symlinks don't take up much space.
>  SOLIBSDEV:darwin = ".dylibbroken"
>
> with the intent of being able to abstract out dylib/so/dll references.
> I'm wondering if we should add:
>
> SOLIBS:mingw32 = ".dll"
> SOLIBSDEV:mingw32 = ".dll"
>
> which would then let us change the binutils.inc entry to:
>
> ${libdir}/bfd-plugins/lib*${SOLIBSDEV}
>
> In testing this I found it breaks zstd and zlib since those always
> generate .so files even on mingw. I've hardcoded SOLIBS and SOLIBSDEV
> in the bbappends in mingw to counteract that:
>
> https://git.yoctoproject.org/meta-mingw/commit/?id=db1f53f73e95499a5f0b0edc45dd9cc9718d3c8a
>
> I think these changes should combine to fix the underlying problem and
> also make it easier to fix things like this in future.
>
> They probably need wider testing though.
>
> Cheers,
>
> Richard
>
>
>
>
>
>
>

On Thu, 2024-09-19 at 14:41 +0100, Richard Purdie via lists.openembedded.org wrote:
> On Thu, 2024-09-19 at 05:48 -0700, Sadineni, Harish via lists.openembedded.org wrote:
> > From: Harish Sadineni <Harish.Sadineni@windriver.com>
> > 
> > Fixes the follwing error while building nativesdk-binutils with i686-mingw32 as SDKMACHINE:
> > ERROR: nativesdk-binutils-2.43.1-r0 do_package: QA Issue: nativesdk-binutils: Files/directories
> > were installed but not shipped in any package:
> >    /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-w64-mingw32/usr/lib/bfd-plugins
> >    /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-w64-mingw32/usr/lib/bfd-plugins/libdep.dll
> > 
> > Signed-off-by: Harish Sadineni <Harish.Sadineni@windriver.com>
> > ---
> >  meta/recipes-devtools/binutils/binutils_2.43.1.bb | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/meta/recipes-devtools/binutils/binutils_2.43.1.bb b/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> > index 2cce40f1ef..eb7c16cbc4 100644
> > --- a/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> > +++ b/meta/recipes-devtools/binutils/binutils_2.43.1.bb
> > @@ -72,5 +72,6 @@ SRC_URI:append:class-nativesdk =  " file://0003-binutils-nativesdk-Search-for-al
> >  
> >  USE_ALTERNATIVES_FOR:class-nativesdk = ""
> >  FILES:${PN}:append:class-nativesdk = " ${bindir}"
> > +FILES:${PN}:append:class-nativesdk = " ${libdir}/bfd-plugins"
> >  
> >  BBCLASSEXTEND = "native nativesdk"
> 
> I ended up digging into this a lot more. I was able to reproduce the
> failure locally but that implies the autobuilder isn't testing this
> combination. According to the configuration, it should be so I'm a bit
> puzzled why the autobuilder doesn't see it.
> 
> The autobuilder config for this is here:
> 
> https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json#n1362
> 
> Resulting in a build such as this:
> 
> https://valkyrie.yoctoproject.org/#/builders/7/builds/162
> 
> I would like to understand why automated testing isn't catching this.

Just to complete things on this topic, I realised the issue. We build
binutils-cross-canadian-* for mingw but nativesdk-bintuils isn't that
useful in general on mingw and isn't built. I'd be curious if you are
actually using it.

Cheers,

Richard

>> Just to complete things on this topic, I realised the issue. We build
>> binutils-cross-canadian-* for mingw but nativesdk-bintuils isn't that
>> useful in general on mingw and isn't built. I'd be curious if you are
>> actually using it.

Although we are not using it, Alexandre Belloni encountered this issue and suggested that my patches might be the cause.
https://autobuilder.yoctoproject.org/typhoon/#/builders/89/builds/8798/steps/12/logs/stdio

After reviewing it, I found that my patches were not responsible, but I am still interested in resolving the issue. That's why I submitted a patch to fix it.

Thanks,
Harish