findutils: split into packages

From: Rasmus Villemoes <rasmus.villemoes@prevas.dk>

It can be useful to have find and/or xargs by themselves without
pulling in locate and its associated binaries updatedb and frcode. In
my case, I just need xargs in an initramfs, which is a 42K binary; the
other binaries add over 300K of dead weight.

So split the recipe into -xargs, -find and -locate packages making it
possible to RDEPEND on just, say, findutils-xargs.

For backwards compatibility, make ${PN} pull in all the new packages.

Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
---
 meta/recipes-extended/findutils/findutils.inc | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

On 17/06/2024 11.59, Rasmus Villemoes wrote:
> 
> For backwards compatibility, make ${PN} pull in all the new packages.
> 
[...]
>  
> +RDEPENDS:${PN} += "${PN}-find ${PN}-xargs ${PN}-locate"

Sorry about not testing properly before sending. For this to work this
also needs

ALLOW_EMPTY:${PN} = "1"

On 17 Jun 2024, at 10:59, Rasmus Villemoes via lists.openembedded.org <rasmus.villemoes=prevas.dk@lists.openembedded.org> wrote:
> 
> From: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
> 
> It can be useful to have find and/or xargs by themselves without
> pulling in locate and its associated binaries updatedb and frcode. In
> my case, I just need xargs in an initramfs, which is a 42K binary; the
> other binaries add over 300K of dead weight.

busybox has both find and xargs, why is that not satisfactory for a initramfs?

Ross

On 17/06/2024 12.22, Ross Burton wrote:
> On 17 Jun 2024, at 10:59, Rasmus Villemoes via lists.openembedded.org <rasmus.villemoes=prevas.dk@lists.openembedded.org> wrote:
>>
>> From: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
>>
>> It can be useful to have find and/or xargs by themselves without
>> pulling in locate and its associated binaries updatedb and frcode. In
>> my case, I just need xargs in an initramfs, which is a 42K binary; the
>> other binaries add over 300K of dead weight.
> 
> busybox has both find and xargs, why is that not satisfactory for a initramfs?

It might be, for this particular use case, and where I control everything.

But, the problem with relying on busybox for anything is that there's no
way to say "depend on busybox built with at least CONFIG_XARGS and
CONFIG_FEATURE_XARGS_SUPPORT_QUOTES". So in order to make the recipe for
my script correct, I'd rather just say "RDEPENDS += findutils-xargs" and
know that I get something with the required features and semantics. I
was bitten by precisely this; I tested whether my script would even work
with busybox xargs on my host machine, was pleasantly surprised that it
did, then tested on target and then realized that the busybox shipped
with Ubuntu apparently has that feature flag set whereas poky's default
busybox config does not.

Apart from the size issue (which of course mostly matters in initramfs
and other constrained environments), there's in general value in being
able to not install binaries that are not needed [*], and apart from
being provided by the same recipe, these binaries don't really have a
lot in common. Yes, 'find -print0 | xargs -0' is a common and useful
thing, but xargs and find can certainly be used by themselves.

[*] Say, when some severe CVE is found in findutils-locate, it's nice to
be able to say "not affected, that binary is not on any of our targets"
despite findutils being in the sbom.

Rasmus

This then needs to happen for all of the core system executables, and not
just these three ones you happen to be using. Packaging should follow a
policy and not be an inconsistent mishmash. But having a policy that splits
every executable into its own package would be a maintenance nightmare.

I’d say this should stay in a private layer if you really don’t want to
just use busybox.

Alex

On Mon 17. Jun 2024 at 13.30, Rasmus Villemoes via lists.openembedded.org
<rasmus.villemoes=prevas.dk@lists.openembedded.org> wrote:

> On 17/06/2024 12.22, Ross Burton wrote:
> > On 17 Jun 2024, at 10:59, Rasmus Villemoes via lists.openembedded.org
> <rasmus.villemoes=prevas.dk@lists.openembedded.org> wrote:
> >>
> >> From: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
> >>
> >> It can be useful to have find and/or xargs by themselves without
> >> pulling in locate and its associated binaries updatedb and frcode. In
> >> my case, I just need xargs in an initramfs, which is a 42K binary; the
> >> other binaries add over 300K of dead weight.
> >
> > busybox has both find and xargs, why is that not satisfactory for a
> initramfs?
>
> It might be, for this particular use case, and where I control everything.
>
> But, the problem with relying on busybox for anything is that there's no
> way to say "depend on busybox built with at least CONFIG_XARGS and
> CONFIG_FEATURE_XARGS_SUPPORT_QUOTES". So in order to make the recipe for
> my script correct, I'd rather just say "RDEPENDS += findutils-xargs" and
> know that I get something with the required features and semantics. I
> was bitten by precisely this; I tested whether my script would even work
> with busybox xargs on my host machine, was pleasantly surprised that it
> did, then tested on target and then realized that the busybox shipped
> with Ubuntu apparently has that feature flag set whereas poky's default
> busybox config does not.
>
> Apart from the size issue (which of course mostly matters in initramfs
> and other constrained environments), there's in general value in being
> able to not install binaries that are not needed [*], and apart from
> being provided by the same recipe, these binaries don't really have a
> lot in common. Yes, 'find -print0 | xargs -0' is a common and useful
> thing, but xargs and find can certainly be used by themselves.
>
> [*] Say, when some severe CVE is found in findutils-locate, it's nice to
> be able to say "not affected, that binary is not on any of our targets"
> despite findutils being in the sbom.
>
> Rasmus
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#200806):
> https://lists.openembedded.org/g/openembedded-core/message/200806
> Mute This Topic: https://lists.openembedded.org/mt/106717870/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

On 17/06/2024 13.43, Alexander Kanavin wrote:
> This then needs to happen for all of the core system executables,

No, that's not at all true, and does not follow logically.

> not just these three ones you happen to be using.

[I happen to need just one, that's the point.]

> Packaging should
> follow a policy and not be an inconsistent mishmash. But having a policy
> that splits every executable into its own package would be a maintenance
> nightmare.

I really don't see how this is any different from any of the dozens of
the existing recipes that show up if you do "git grep 'FILES.*bindir'".
iproute2, e2fsprogs, psmisc, openssh, mtd-utils and lots of others
package binaries in distinct packages (not necessarily
one-executable-per-package, e.g. mtd-utils collects the ubifs and jffs2
related ones).

> I’d say this should stay in a private layer if you really don’t want to
> just use busybox.

I already explained why relying on busybox providing the necessary
applet with the necessary features is the inferior option. I also
provided additional arguments for why allowing one to install, say,
xargs, without pulling in the locate binary, might be desirable.

Rasmus

On Mon, 17 Jun 2024 at 14:29, Rasmus Villemoes
<rasmus.villemoes@prevas.dk> wrote:
> I really don't see how this is any different from any of the dozens of
> the existing recipes that show up if you do "git grep 'FILES.*bindir'".
> iproute2, e2fsprogs, psmisc, openssh, mtd-utils and lots of others
> package binaries in distinct packages (not necessarily
> one-executable-per-package, e.g. mtd-utils collects the ubifs and jffs2
> related ones).

Yes, there are a few recipes that do this (not dozens). There are far
more recipes that do not.

The given reasoning is essentially 'I can't be bothered to configure
busybox correctly and test my script with it in a systematic way',
which does rub me the wrong way. Usually executables are split to
avoid licensing issues in a product, avoid installing both client and
server, or to satisfy space constraints if there's no smaller
alternative - and in this case there is.

A change to the default busybox configuration in oe-core is a better
way out perhaps? Enable the needed options, and send a patch.

Alex