Message ID | 20221027231350.418471-1-bero@baylibre.com |
---|---|
State | New |
Headers | show |
Series | base.bbclass: Try harder to find a compatible tar implementation | expand |
Interesting. I think the more common approach is just to compile a native tar with Yocto and add it with EXTRANATIVEPATH (iirc), like suggested here: https://stackoverflow.com/a/43109887 Regards //Ernst Den fre 28 okt. 2022 kl 01:14 skrev Bernhard Rosenkränzer via lists.openembedded.org <bero=lindev.ch@lists.openembedded.org>: > From: Bernhard Rosenkränzer <bero@baylibre.com> > > base.bbclass currently symlinks the first copy of tar on the PATH > to hosttools. > > If that copy of tar isn't GNU tar (some Linux distributions use > libarchive tar for the fact that it can handle zip files etc. as > well; some others probably use busybox or toybox tar for space > issues), the build fails early on because path.py uses --xattrs > and --xattrs-include > > With this patch, base.bbclass checks if tar supports --xattrs-include, > and checks for gtar if the first tar found doesn't support it. > > Signed-off-by: Bernhard Rosenkränzer <bero@baylibre.com> > --- > meta/classes-global/base.bbclass | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff --git a/meta/classes-global/base.bbclass > b/meta/classes-global/base.bbclass > index 8203f54519..b6391501ca 100644 > --- a/meta/classes-global/base.bbclass > +++ b/meta/classes-global/base.bbclass > @@ -135,6 +135,16 @@ def setup_hosttools_dir(dest, toolsvar, d, > fatal=True): > if os.path.islink(desttool): > os.unlink(desttool) > srctool = bb.utils.which(path, tool, executable=True) > + # copytree() uses tar options such as --xattrs and > + # --xattrs-include that are currently specific to GNU tar. > + # If the first tar in PATH is libarchive tar or busybox/toybox > + # tar, GNU tar may well be available as gtar. > + if (tool == "tar"): > + import subprocess > + if not b'--xattrs-include' in > subprocess.check_output([srctool, '--help']): > + gtar = bb.utils.which(path, "gtar", executable=True) > + if gtar: > + srctool = gtar > # gcc/g++ may link to ccache on some hosts, e.g., > # /usr/local/bin/ccache/gcc -> /usr/bin/ccache, then > which(gcc) > # would return /usr/local/bin/ccache/gcc, but what we need is > -- > 2.38.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#172229): > https://lists.openembedded.org/g/openembedded-core/message/172229 > Mute This Topic: https://lists.openembedded.org/mt/94616037/4947266 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > ernstp@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On 28 Oct 2022, at 00:13, Bernhard Rosenkränzer via lists.openembedded.org <bero=lindev.ch@lists.openembedded.org> wrote: > If that copy of tar isn't GNU tar (some Linux distributions use > libarchive tar for the fact that it can handle zip files etc. as > well; some others probably use busybox or toybox tar for space > issues), the build fails early on because path.py uses --xattrs > and --xattrs-include Can we name what linux distribution that is? Ross
On Fri, 2022-10-28 at 01:13 +0200, Bernhard Rosenkränzer via lists.openembedded.org wrote: > From: Bernhard Rosenkränzer <bero@baylibre.com> > > base.bbclass currently symlinks the first copy of tar on the PATH > to hosttools. > > If that copy of tar isn't GNU tar (some Linux distributions use > libarchive tar for the fact that it can handle zip files etc. as > well; some others probably use busybox or toybox tar for space > issues), the build fails early on because path.py uses --xattrs > and --xattrs-include > > With this patch, base.bbclass checks if tar supports --xattrs-include, > and checks for gtar if the first tar found doesn't support it. > > Signed-off-by: Bernhard Rosenkränzer <bero@baylibre.com> > --- > meta/classes-global/base.bbclass | 10 ++++++++++ > 1 file changed, 10 insertions(+) This feels like a rather dangerous path to start down. HOSTTOOLS is there for host isolation, not to remap the tools to different things depending on the distro we're on or versions etc. I can see the attraction in using it to manipulate it but I'd kind of prefer to error if tar isn't compatible and allow the user to change their distro setup rather than magic... Cheers, Richard
On 28 Oct 2022, at 12:41, Richard Purdie via lists.openembedded.org <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > This feels like a rather dangerous path to start down. > > HOSTTOOLS is there for host isolation, not to remap the tools to > different things depending on the distro we're on or versions etc. > > I can see the attraction in using it to manipulate it but I'd kind of > prefer to error if tar isn't compatible and allow the user to change > their distro setup rather than magic… Patch incoming to detect a tar that doesn’t support —xattrs and abort early, instead of mysteriously later. Ross
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass index 8203f54519..b6391501ca 100644 --- a/meta/classes-global/base.bbclass +++ b/meta/classes-global/base.bbclass @@ -135,6 +135,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True): if os.path.islink(desttool): os.unlink(desttool) srctool = bb.utils.which(path, tool, executable=True) + # copytree() uses tar options such as --xattrs and + # --xattrs-include that are currently specific to GNU tar. + # If the first tar in PATH is libarchive tar or busybox/toybox + # tar, GNU tar may well be available as gtar. + if (tool == "tar"): + import subprocess + if not b'--xattrs-include' in subprocess.check_output([srctool, '--help']): + gtar = bb.utils.which(path, "gtar", executable=True) + if gtar: + srctool = gtar # gcc/g++ may link to ccache on some hosts, e.g., # /usr/local/bin/ccache/gcc -> /usr/bin/ccache, then which(gcc) # would return /usr/local/bin/ccache/gcc, but what we need is