base.bbclass: Try harder to find a compatible tar implementation

From: Bernhard Rosenkränzer <bero@baylibre.com>

base.bbclass currently symlinks the first copy of tar on the PATH
to hosttools.

If that copy of tar isn't GNU tar (some Linux distributions use
libarchive tar for the fact that it can handle zip files etc. as
well; some others probably use busybox or toybox tar for space
issues), the build fails early on because path.py uses --xattrs
and --xattrs-include

With this patch, base.bbclass checks if tar supports --xattrs-include,
and checks for gtar if the first tar found doesn't support it.

Signed-off-by: Bernhard Rosenkränzer <bero@baylibre.com>
---
 meta/classes-global/base.bbclass | 10 ++++++++++
 1 file changed, 10 insertions(+)

Interesting. I think the more common approach is just to compile a native
tar with Yocto and add it with EXTRANATIVEPATH (iirc),
like suggested here: https://stackoverflow.com/a/43109887

Regards
//Ernst

Den fre 28 okt. 2022 kl 01:14 skrev Bernhard Rosenkränzer via
lists.openembedded.org <bero=lindev.ch@lists.openembedded.org>:

> From: Bernhard Rosenkränzer <bero@baylibre.com>
>
> base.bbclass currently symlinks the first copy of tar on the PATH
> to hosttools.
>
> If that copy of tar isn't GNU tar (some Linux distributions use
> libarchive tar for the fact that it can handle zip files etc. as
> well; some others probably use busybox or toybox tar for space
> issues), the build fails early on because path.py uses --xattrs
> and --xattrs-include
>
> With this patch, base.bbclass checks if tar supports --xattrs-include,
> and checks for gtar if the first tar found doesn't support it.
>
> Signed-off-by: Bernhard Rosenkränzer <bero@baylibre.com>
> ---
>  meta/classes-global/base.bbclass | 10 ++++++++++
>  1 file changed, 10 insertions(+)
>
> diff --git a/meta/classes-global/base.bbclass
> b/meta/classes-global/base.bbclass
> index 8203f54519..b6391501ca 100644
> --- a/meta/classes-global/base.bbclass
> +++ b/meta/classes-global/base.bbclass
> @@ -135,6 +135,16 @@ def setup_hosttools_dir(dest, toolsvar, d,
> fatal=True):
>              if os.path.islink(desttool):
>                  os.unlink(desttool)
>              srctool = bb.utils.which(path, tool, executable=True)
> +            # copytree() uses tar options such as --xattrs and
> +            # --xattrs-include that are currently specific to GNU tar.
> +            # If the first tar in PATH is libarchive tar or busybox/toybox
> +            # tar, GNU tar may well be available as gtar.
> +            if (tool == "tar"):
> +                import subprocess
> +                if not b'--xattrs-include' in
> subprocess.check_output([srctool, '--help']):
> +                    gtar = bb.utils.which(path, "gtar", executable=True)
> +                    if gtar:
> +                        srctool = gtar
>              # gcc/g++ may link to ccache on some hosts, e.g.,
>              # /usr/local/bin/ccache/gcc -> /usr/bin/ccache, then
> which(gcc)
>              # would return /usr/local/bin/ccache/gcc, but what we need is
> --
> 2.38.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#172229):
> https://lists.openembedded.org/g/openembedded-core/message/172229
> Mute This Topic: https://lists.openembedded.org/mt/94616037/4947266
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> ernstp@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

On 28 Oct 2022, at 00:13, Bernhard Rosenkränzer via lists.openembedded.org <bero=lindev.ch@lists.openembedded.org> wrote:
> If that copy of tar isn't GNU tar (some Linux distributions use
> libarchive tar for the fact that it can handle zip files etc. as
> well; some others probably use busybox or toybox tar for space
> issues), the build fails early on because path.py uses --xattrs
> and --xattrs-include

Can we name what linux distribution that is?

Ross

On Fri, 2022-10-28 at 01:13 +0200, Bernhard Rosenkränzer via
lists.openembedded.org wrote:
> From: Bernhard Rosenkränzer <bero@baylibre.com>
> 
> base.bbclass currently symlinks the first copy of tar on the PATH
> to hosttools.
> 
> If that copy of tar isn't GNU tar (some Linux distributions use
> libarchive tar for the fact that it can handle zip files etc. as
> well; some others probably use busybox or toybox tar for space
> issues), the build fails early on because path.py uses --xattrs
> and --xattrs-include
> 
> With this patch, base.bbclass checks if tar supports --xattrs-include,
> and checks for gtar if the first tar found doesn't support it.
> 
> Signed-off-by: Bernhard Rosenkränzer <bero@baylibre.com>
> ---
>  meta/classes-global/base.bbclass | 10 ++++++++++
>  1 file changed, 10 insertions(+)

This feels like a rather dangerous path to start down.

HOSTTOOLS is there for host isolation, not to remap the tools to
different things depending on the distro we're on or versions etc.

I can see the attraction in using it to manipulate it but I'd kind of
prefer to error if tar isn't compatible and allow the user to change
their distro setup rather than magic...

Cheers,

Richard

On 28 Oct 2022, at 12:41, Richard Purdie via lists.openembedded.org <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote:
> This feels like a rather dangerous path to start down.
> 
> HOSTTOOLS is there for host isolation, not to remap the tools to
> different things depending on the distro we're on or versions etc.
> 
> I can see the attraction in using it to manipulate it but I'd kind of
> prefer to error if tar isn't compatible and allow the user to change
> their distro setup rather than magic…

Patch incoming to detect a tar that doesn’t support —xattrs and abort early, instead of mysteriously later.

Ross