diff mbox series

[bitbake-devel] cooker: Add layer-specific ignore for dangling appends

Message ID 20250729144906.1665908-1-JPEWhacker@gmail.com
State New
Headers show
Series [bitbake-devel] cooker: Add layer-specific ignore for dangling appends | expand

Commit Message

Joshua Watt July 29, 2025, 2:47 p.m. UTC
Implements a new layer-specific variable called
BB_DANGLINGAPPENDS_IGNORE. This variable is similar to the removed
BB_DANGLINGAPPENDS_WARNONLY, except that it is scoped to a specific
layer, and it also causes appends to be completely ignored instead of
warning about them.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
BACKGROUND AND RATIONAL:

I was fully on board with removing BB_DANGLINGAPPENDS_WARNONLY, since it
was pretty bad that it operated on all layers if any layer set the
variable. However, I did mention at the time that it was going to cause
us some trouble, but not wanting to hold up a good change, I indicated
that I would attempt a fix for it later. Later is now :)

After much poking around in our setup and deliberation, the conclusion
that I came to is that we do actually need _something_ along the lines
of being able to ignore dangling appends, at least in a few specific
layers.

Our setup is that we have a "disto" layer that we need to maintain
against several version of Yocto at once. In this disto layer, we make
policy choices that sometimes are tied to a specific version of a recipe
found in a specific version of Yocto using a bbappend; and often that
version is only in one or two Yocto versions, which means that the
bbappend will be dangling in the others.

dynamic-layers doesn't work here because we are keying off the Yocto
version, not the existence of some layer (unless there is something I'm
unaware of).

BBMASK is also a less than satisfactory solution (IMHO). Part of the
reason we have structured our distro layer in the way we have is to
reduce the cognitive overhead required to maintain our dozens of
products that more or less need to behave similarly. Keeping all the
distro stuff together makes the quite easy to reason about, but having
then again to track (probably quite complicated, and possibly
per-product) BBMASKs erodes that maintainability.

We did go through and eliminate as many dangling appends with other
methods as possible, but at the end of the day, we ended up with some
for which we didn't really see a viable (and maintainable) solution.

I'm open to suggestions if this change is entirely reprehensible, but
our primary goal is to keep the maintenance of our cross-yocto distro
layer simple.

I also decided to simplify this to just ignore dangling BBAPPENDS, since
I suspect anyone setting this is simply going to ignore the warnings
anyway, but that can be changed if there's a strong option to do it
differently.

 bitbake/lib/bb/cooker.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

Comments

Alexander Kanavin July 29, 2025, 2:57 p.m. UTC | #1
On Tue, 29 Jul 2025 at 16:49, Joshua Watt via lists.openembedded.org
<JPEWhacker=gmail.com@lists.openembedded.org> wrote:
> Our setup is that we have a "disto" layer that we need to maintain
> against several version of Yocto at once. In this disto layer, we make
> policy choices that sometimes are tied to a specific version of a recipe
> found in a specific version of Yocto using a bbappend; and often that
> version is only in one or two Yocto versions, which means that the
> bbappend will be dangling in the others.

This is where the weak point of the argument is. Yocto does not make
concessions to layers that attempt to be compatible across yocto
releases, patches that help such layers had been previously rejected,
and this would create a precedent to the contrary.

It's always possible to make a branch.

Alex
Joshua Watt July 29, 2025, 2:59 p.m. UTC | #2
On Tue, Jul 29, 2025 at 8:57 AM Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> On Tue, 29 Jul 2025 at 16:49, Joshua Watt via lists.openembedded.org
> <JPEWhacker=gmail.com@lists.openembedded.org> wrote:
> > Our setup is that we have a "disto" layer that we need to maintain
> > against several version of Yocto at once. In this disto layer, we make
> > policy choices that sometimes are tied to a specific version of a recipe
> > found in a specific version of Yocto using a bbappend; and often that
> > version is only in one or two Yocto versions, which means that the
> > bbappend will be dangling in the others.
>
> This is where the weak point of the argument is. Yocto does not make
> concessions to layers that attempt to be compatible across yocto
> releases, patches that help such layers had been previously rejected,
> and this would create a precedent to the contrary.
>
> It's always possible to make a branch.
>

Branching isn't an option for us; we are unwilling to add the maintenance
overhead of dealing with that.


>
> Alex
>
Richard Purdie July 29, 2025, 10:42 p.m. UTC | #3
On Tue, 2025-07-29 at 08:47 -0600, Joshua Watt via lists.openembedded.org wrote:
> Implements a new layer-specific variable called
> BB_DANGLINGAPPENDS_IGNORE. This variable is similar to the removed
> BB_DANGLINGAPPENDS_WARNONLY, except that it is scoped to a specific
> layer, and it also causes appends to be completely ignored instead of
> warning about them.
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
> BACKGROUND AND RATIONAL:
> 
> I was fully on board with removing BB_DANGLINGAPPENDS_WARNONLY, since it
> was pretty bad that it operated on all layers if any layer set the
> variable. However, I did mention at the time that it was going to cause
> us some trouble, but not wanting to hold up a good change, I indicated
> that I would attempt a fix for it later. Later is now :)
> 
> After much poking around in our setup and deliberation, the conclusion
> that I came to is that we do actually need _something_ along the lines
> of being able to ignore dangling appends, at least in a few specific
> layers.
> 
> Our setup is that we have a "disto" layer that we need to maintain
> against several version of Yocto at once. In this disto layer, we make
> policy choices that sometimes are tied to a specific version of a recipe
> found in a specific version of Yocto using a bbappend; and often that
> version is only in one or two Yocto versions, which means that the
> bbappend will be dangling in the others.
> 
> dynamic-layers doesn't work here because we are keying off the Yocto
> version, not the existence of some layer (unless there is something I'm
> unaware of).
> 
> BBMASK is also a less than satisfactory solution (IMHO). Part of the
> reason we have structured our distro layer in the way we have is to
> reduce the cognitive overhead required to maintain our dozens of
> products that more or less need to behave similarly. Keeping all the
> distro stuff together makes the quite easy to reason about, but having
> then again to track (probably quite complicated, and possibly
> per-product) BBMASKs erodes that maintainability.
> 
> We did go through and eliminate as many dangling appends with other
> methods as possible, but at the end of the day, we ended up with some
> for which we didn't really see a viable (and maintainable) solution.
> 
> I'm open to suggestions if this change is entirely reprehensible, but
> our primary goal is to keep the maintenance of our cross-yocto distro
> layer simple.
> 
> I also decided to simplify this to just ignore dangling BBAPPENDS, since
> I suspect anyone setting this is simply going to ignore the warnings
> anyway, but that can be changed if there's a strong option to do it
> differently.
> 
>  bitbake/lib/bb/cooker.py | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
> index 1810bcc604..349e27d538 100644
> --- a/bitbake/lib/bb/cooker.py
> +++ b/bitbake/lib/bb/cooker.py
> @@ -953,7 +953,11 @@ You can also remove the BB_HASHSERVE_UPSTREAM setting, but this may result in si
>              appends_without_recipes[mc] = []
>              for _, appendfn in self.collections[mc].bbappends:
>                  if not appendfn in applied_appends:
> -                    appends_without_recipes[mc].append(appendfn)
> +                    layername = self.collections[mc].calc_bbfile_priority(appendfn)[2]
> +                    ignore = self.databuilder.mcdata[mc].getVar("BB_DANGLINGAPPENDS_IGNORE_%s" % layername,
> +                        False) or "no"
> +                    if ignore.lower() not in ("1", "yes", "true"):

At least use bb.utils.to_boolean() so this behaves the same as any
other true/false value we have.

> +                        appends_without_recipes[mc].append(appendfn)

I'm torn on this. On the one hand I do want to support our user base,
on the other this is adding back some API we decided we really didn't
want. I appreciate it is different in that it is layer specific and it
is silent rather than warning, but I'm not convinced either of those
things sell me on the idea that we want to encourage people to be doing
this. We did consider making this layer specific and decided against
that at the time it was removed.

The choice here is between defined and explicit configurations and
"mashing a load of stuff together" and hoping it works. We directed
people to BBMASK as that is much more explicit. I suspect with some
distro version overrides, BBMASK can be made to work but it will be
more effort that just ignoring any mismatches.

Cheers,

Richard
Zach Welch July 30, 2025, 11:12 p.m. UTC | #4
On 7/29/25 07:59, Joshua Watt via lists.openembedded.org wrote:
> 
> 
> On Tue, Jul 29, 2025 at 8:57 AM Alexander Kanavin 
> <alex.kanavin@gmail.com <mailto:alex.kanavin@gmail.com>> wrote:
> 
>     On Tue, 29 Jul 2025 at 16:49, Joshua Watt via lists.openembedded.org
>     <http://lists.openembedded.org>
>     <JPEWhacker=gmail.com@lists.openembedded.org
>     <mailto:gmail.com@lists.openembedded.org>> wrote:
>      > Our setup is that we have a "disto" layer that we need to maintain
>      > against several version of Yocto at once. In this disto layer, we
>     make
>      > policy choices that sometimes are tied to a specific version of a
>     recipe
>      > found in a specific version of Yocto using a bbappend; and often that
>      > version is only in one or two Yocto versions, which means that the
>      > bbappend will be dangling in the others.
> 
>     This is where the weak point of the argument is. Yocto does not make
>     concessions to layers that attempt to be compatible across yocto
>     releases, patches that help such layers had been previously rejected,
>     and this would create a precedent to the contrary.
> 
>     It's always possible to make a branch.
> 
> 
> Branching isn't an option for us; we are unwilling to add the 
> maintenance overhead of dealing with that.

I have resorted to some dirty tricks to solve problems like this in the 
past with great success.

If you move the incompatible bbappends into their own '$branch' 
directories in your distro layer, then you can add lines like this to 
your `layer.conf`:

   BBFILES += "${@ "${LAYERDIR}/$branch/recipes-*/*/*.bbappend" if 
d.getVar("DISTRO_CODENAME") == "$branch" else "" }"

I just tested that this trick does appear to work.  But as no one has 
jumped in to offer this solution before now, I suspect that this 
actually might be a bad idea for... reasons?

Cheers,
Zach
Chuck Wolber Aug. 9, 2025, 7:59 p.m. UTC | #5
On Tue Jul 29, 2025 at 7:59 AM PDT, Joshua Watt via lists.openembedded.org
wrote:
> On Tue, Jul 29, 2025 at 8:57 AM Alexander Kanavin <alex.kanavin@gmail.com>
> wrote:
>
>> On Tue, 29 Jul 2025 at 16:49, Joshua Watt via lists.openembedded.org
>> <JPEWhacker=gmail.com@lists.openembedded.org> wrote:
>> >
>> > Our setup is that we have a "disto" layer that we need to maintain against
>> > several version of Yocto at once. In this disto layer, we make policy
>> > choices that sometimes are tied to a specific version of a recipe found in
>> > a specific version of Yocto using a bbappend; and often that version is
>> > only in one or two Yocto versions, which means that the bbappend will be
>> > dangling in the others.
>>
>> This is where the weak point of the argument is. Yocto does not make
>> concessions to layers that attempt to be compatible across yocto releases,
>> patches that help such layers had been previously rejected, and this would
>> create a precedent to the contrary.

Yes, but perhaps some consideration is merited? This is a perennial problem for
OEMs with long lived products.


>> It's always possible to make a branch.

While this is logically correct, it is also an unreasonable response.

The argument is being made with sincerity, clarity, and consistency. I think it
there is a lot to be gained by taking more time to understand it.


> Branching isn't an option for us; we are unwilling to add the maintenance
> overhead of dealing with that.

You are not alone in this respect. Perhaps we need to find a better way to
explain this to the YP community? I believe adequate consideration of the
problem will improve more than just the plight of OEMs with long lived
products.


..Ch:W..
Alexander Kanavin Aug. 10, 2025, 2:26 p.m. UTC | #6
On Sat, 9 Aug 2025 at 21:59, Chuck Wolber <chuck@wolber.net> wrote:

> > Branching isn't an option for us; we are unwilling to add the maintenance
> > overhead of dealing with that.
>
> You are not alone in this respect. Perhaps we need to find a better way to
> explain this to the YP community? I believe adequate consideration of the
> problem will improve more than just the plight of OEMs with long lived
> products.

I am totally listening, but you (and Joshua) aren't actually saying much.

You do need to explain with sincerity and clarity and full details
what is this 'maintenance overhead' that is so burdensome that
branching becomes impossible. Pretty much every community layer
manages this, so why not your product layer?

Alex
Joshua Watt Aug. 11, 2025, 3:49 p.m. UTC | #7
> I am totally listening, but you (and Joshua) aren't actually saying much.
>
> You do need to explain with sincerity and clarity and full details
> what is this 'maintenance overhead' that is so burdensome that
> branching becomes impossible. Pretty much every community layer
> manages this, so why not your product layer?
>
>
Ok.

THE BACKGROUND:

We don't produce just one product, we produce many products simultaneously
(north of 60), and we do so at scale. One of the things we highly value is
the efficiency with which we can develop so many products at the same time.
Developer efficiency is one of the primary reasons that we chose to use
Yocto in the first place; while the first product we did had a bit of a
steep learning curve, it was very clear that we could efficiently scale it
out much better than competitors. Looking back, I think the project has
actually exceeded my expectations for just how well we've been able to
scale.

Highly valuing efficient with our development has a few side effects:

 1) We do (more or less) trunk development on all our products at once.
This means there is a single branch where we do all development work on all
our products simultaneously, regardless of what version of upstream Yocto
they are using, which BSP they are using etc. This not only means its easy
for developers to find all products, but our CI also tests any change made
against all products to make sure it doesn't break anything. We aren't
willing to give up this development style and trade away the efficiencies
this provides, which is why we can't simply branch our internal repos.

 2) We like all our products to behave consistently, at least in userspace
if not sooner. This means that as much as possible we configure our distro
to act the same, regardless of Yocto release, BSP, etc. Similarly, if the
BSP allows it, we like to use the same boot flow across all products, etc.
Consistency allows our userspace developers to be more efficient because
they know that the code they write should work the same regardless of which
platform it's running on, the development and debugging experience is
pretty much the same across all products, etc. It (mostly) eliminates the
need for a developer to have to learn 60+ different ways of doing
development. However, we still need to customize the behavior of many
things (think distro policy settings, etc.), so we need a way to make sure
we get as similar behavior in these customization as possible, even across
multiple Yocto releases.

 3) We don't diverge from master of upstream OE (or any other layer). We
have internal mirrors of all the layers we use, and we have a branch for
each Yocto version in those mirrors that we maintain. I will apply changes
to these internal mirror branches for a given release (effectively forking
it), but I *never* fork master (e.g. master always exactly matches upstream
master). Any change that is made to a release branch in the mirror I
require to exist on the master branch first (this is occasionally
unpopular). The reason for this is so that as we move to newer releases, we
aren't carrying along patches that are not upstreamed. Another way to think
about this is that I'll more or less backport anything to a release branch,
as long as the behavior matches master. I wouldn't say this is entirely
ideal, but it does IMHO strike the proper balance between allowing us to do
trunk development and not diverging from upstream in the long run.

THE PROBLEM:

Hopefully, that sets the stage for *why* we do things the way that we do,
and now we can look a concrete example of why we'd prefer to allow dangling
bbappends. Here is the list of bbappend we use in our disto layer for
wpa-supplicant:

  wpa-supplicant_2.%.bbappend
  wpa-supplicant_2.10.bbappend
  wpa-supplicant_2.11.bbappend
  wpa-supplicant_2.5.bbappend
  wpa-supplicant_2.6.bbappend
  wpa-supplicant_2.7.bbappend
  wpa-supplicant_2.9.bbappend
  wpa-supplicant-CHIP.bbappend

The primary purpose here is to configure wpa-supplicant based on our
product requirements, and remember that we want consistent behavior of
wpa-supplicant across all products. As such, we put as much of the
configuration as we can into wpa-supplication_2.%.bbappend so that it
applies to all wpa-supplicants. However, it's simply not possible to use
the same configuration across all wpa-supplicant versions; new options are
added, old one removed etc. so some configuration changes can only be
applied to specific version, or some versions need a patch to behave like
the latest version, etc. Finally, some BSPs choose to provide their own
wpa-supplicant recipe, and those have to be configured also, which is what
the wpa-supplicant-CHIP.bbappend stands for.

From the perspective of dangling bbappends, this is awful. However, from
the perspective of our developers, this is pretty efficient. Our developers
can go to a single directory and see *all* the distro changes for all
versions of wpa-supplicant. If multiple versions need to share a config
file (this happens frequently), they can easily do that since they all live
in the same directory. It's much easier for our developers to make correct
changes, and understand the implication of changes made in this directory.
Overall, we've been happy to trade off dangling bbappends for this.

A POSSIBLE SOLUTION:

For completeness, since it has appeared that upstream is unwilling to
accommodate this, we have tried to work around this. The approach is to
make our conf/layer.conf look like this:

BBFILES += " \
     ${LAYERDIR}/recipes-*/*/*.bb \
     ${LAYERDIR}/recipes-*/*/*.bbappend \
     ${LAYERDIR}/yocto-${LAYERSERIES_CORENAMES}/recipes-*/*/*.bbappend \
     "

Effectively placing the Yocto version specific bbappends into their own
directory. However, this alone is insufficient because we also have to
account for dangling bbappends in layers that might not be present in a
given product (e.g. because it doesn't use a specific BSP). This
effectively means using dynamic layers is forced in order to prevent
dangling bbappends, however it gets worse when you have dynamic layers that
have version specific bbappends, because then we end up with something like
this:

BBFILES_DYNAMIC += " \
    raspberrypi:${LAYERDIR}/dynamic-layers/raspberrypi/*/*/*.bb \
    raspberrypi:${LAYERDIR}/dynamic-layers/raspberrypi/*/*/*.bbappend \

raspberrypi:${LAYERDIR}/dynamic-layers/yocto-${LAYERSERIES_CORENAMES}/raspberrypi/*/*/*.bbappend
\
    "

This approach (or other similar to it) are certainly better than branching
since we can still do trunk development, but what this effectively doing is
placing each bbappend in it's own directory, so instead of looking in a
single place to understand how we are configuring wpa-supplicant, we now
must look in no less than 8 places, and probably more since we'd have to
look in all the places a bbappend _might_ be instead of all the places that
_actually_ have one. Also, doing this makes it much more difficult to share
a common config file across multiple versions, since the bbappends are all
in separate directories.

This might be what we'll do if upstream refuses to allow the dangling
bbappends.... but we'd still just prefer to have the dangling bbappends as
the lesser of two evils.

FINAL THOUGHTS:

Looking behind the curtain a little, my struggle with this is that there is
some pressure to fork master (and break rule #3) so that we can continue to
have the developer efficiency we are accustom to, so I'm probably going to
have some hard conversions about where we go from here. That's not really
upstreams problem, but I wanted to shed some light on the ramifications of
these types of changes.

Hopefully that accurately outlines our struggle here and why we would like
this feature.

Thanks
Joshua Watt
Richard Purdie Aug. 12, 2025, 9:29 a.m. UTC | #8
On Sat, 2025-08-09 at 12:59 -0700, Chuck Wolber via lists.openembedded.org wrote:
> On Tue Jul 29, 2025 at 7:59 AM PDT, Joshua Watt via lists.openembedded.org
> wrote:
> > On Tue, Jul 29, 2025 at 8:57 AM Alexander Kanavin <alex.kanavin@gmail.com>
> > wrote:
> > 
> > > On Tue, 29 Jul 2025 at 16:49, Joshua Watt via lists.openembedded.org
> > > <JPEWhacker=gmail.com@lists.openembedded.org> wrote:
> > > > 
> > > > Our setup is that we have a "disto" layer that we need to maintain against
> > > > several version of Yocto at once. In this disto layer, we make policy
> > > > choices that sometimes are tied to a specific version of a recipe found in
> > > > a specific version of Yocto using a bbappend; and often that version is
> > > > only in one or two Yocto versions, which means that the bbappend will be
> > > > dangling in the others.
> > > 
> > > This is where the weak point of the argument is. Yocto does not make
> > > concessions to layers that attempt to be compatible across yocto releases,
> > > patches that help such layers had been previously rejected, and this would
> > > create a precedent to the contrary.
> 
> Yes, but perhaps some consideration is merited? This is a perennial problem for
> OEMs with long lived products.
> 
> 
> > > It's always possible to make a branch.
> 
> While this is logically correct, it is also an unreasonable response.
> 
> The argument is being made with sincerity, clarity, and consistency. I think it
> there is a lot to be gained by taking more time to understand it.
> 
> 
> > Branching isn't an option for us; we are unwilling to add the maintenance
> > overhead of dealing with that.
> 
> You are not alone in this respect. Perhaps we need to find a better way to
> explain this to the YP community? I believe adequate consideration of the
> problem will improve more than just the plight of OEMs with long lived
> products.

We took a decision to drop dangling bbappend support. We did that for
very specific reasons after discussion at the time. It wasn't a
universally popular decision but the issues that they caused and the 
improvements dropping them allowed seemed attractive. The decision was
made and acted upon.

Now, we're being asked to effectively reverse that decision (with a
slightly different implementation we could have switched to at the time
but chose not to).

It is clear that the usecase this is being asked for is not something
we've officially supported, ever, and that has been really clear.

The precedent here is problematic if I do take this, since whether we
officially say so or not, we're effectively unofficially supporting
that use case.

We have two YP platinum members saying "we really want this".
Ironically we also have the wider YP membership telling me to say "no"
more often and more forcefully.

To me, this isn't an issue of discussing the pros/cons as personally,
I'm well aware of the trade offs on both sides of this.

Rather than the technical merits, the main concern for me is that I do
want to encourage project adoption and use and to push back against
this will hurt that and also potentially hurt some of our key
developers standing within their companies. As such, I don't feel like
I may have a lot of choice but to take this, but perhaps not for the
reasons people expect.

Cheers,

Richard
Alexander Kanavin Aug. 15, 2025, 10:33 a.m. UTC | #9
On Mon, 11 Aug 2025 at 17:49, Joshua Watt <jpewhacker@gmail.com> wrote:

> Hopefully that accurately outlines our struggle here and why we would like this feature.

Thanks for the detailed write-up Joshua. Given what RP said in this
thread after this email appeared, I'd like to abstain from further
commenting.

Alex
Alexander Kanavin Aug. 15, 2025, 11:10 a.m. UTC | #10
On Fri, 15 Aug 2025 at 12:33, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
wrote:
> > Hopefully that accurately outlines our struggle here and why we would like this feature.
>
> Thanks for the detailed write-up Joshua. Given what RP said in this
> thread after this email appeared, I'd like to abstain from further
> commenting.

Famous last words :)

After I sent it an idea popped into my head:

why not have two layers:

- one layer that exists for the sole purpose of hosting bbappends, and
yes it would have branches for each of the respective yocto release.
You'd have to devise a process for patch flow that ensures bbappend
tweaks get applied on every affected branch. There's also a benefit in
having less file clutter, as any branch would contain only files that
are directly relevant and used, and one wouldn't have to figure out
which bbappend goes into which product.

- another layer that contains everything else, including distro
definitions, and product-related recipes, and has only a single trunk
branch like now.

Would that be still unworkable and cause too much overhead? I can't
imagine that bbappends get a lot of churn and are changed often.

Alex
Peter Kjellerstedt Aug. 17, 2025, 8:34 p.m. UTC | #11
> -----Original Message-----
> From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> On Behalf Of Alexander Kanavin via lists.openembedded.org
> Sent: den 15 augusti 2025 13:10
> To: alex.kanavin@gmail.com
> Cc: Joshua Watt <jpewhacker@gmail.com>; Chuck Wolber <chuck@wolber.net>; bitbake-devel@lists.openembedded.org
> Subject: Re: [bitbake-devel][PATCH] cooker: Add layer-specific ignore for dangling appends
> 
> On Fri, 15 Aug 2025 at 12:33, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> > > Hopefully that accurately outlines our struggle here and why we would like this feature.
> >
> > Thanks for the detailed write-up Joshua. Given what RP said in this
> > thread after this email appeared, I'd like to abstain from further
> > commenting.
> 
> Famous last words :)
> 
> After I sent it an idea popped into my head:
> 
> why not have two layers:
> 
> - one layer that exists for the sole purpose of hosting bbappends, and
> yes it would have branches for each of the respective yocto release.
> You'd have to devise a process for patch flow that ensures bbappend
> tweaks get applied on every affected branch. There's also a benefit in
> having less file clutter, as any branch would contain only files that
> are directly relevant and used, and one wouldn't have to figure out
> which bbappend goes into which product.
> 
> - another layer that contains everything else, including distro
> definitions, and product-related recipes, and has only a single trunk
> branch like now.
> 
> Would that be still unworkable and cause too much overhead? I can't
> imagine that bbappends get a lot of churn and are changed often.
> 
> Alex

That is basically what we do. We have our normal distro/application layers. 
Their master branches work with the latest Poky release (currently Walnascar 
5.2.1, but soon 5.2.2). Then to build with some other version of Poky (in 
our case only with master of Poky), we have a separate layer where we do 
all adaptations. This layer typically has a long list of BBMASKs to drop 
the versioned bbappends that are in our application layers, either because 
they are no longer needed, or updated versions of the bbappends are in the 
adaptation layer. When it is time to upgrade to a new version of Poky, all 
the changes in the adaptation layer are applied to the real layers, and 
the adaptation layer is cleaned out, starting a new cycle. 

//Peter
diff mbox series

Patch

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index 1810bcc604..349e27d538 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -953,7 +953,11 @@  You can also remove the BB_HASHSERVE_UPSTREAM setting, but this may result in si
             appends_without_recipes[mc] = []
             for _, appendfn in self.collections[mc].bbappends:
                 if not appendfn in applied_appends:
-                    appends_without_recipes[mc].append(appendfn)
+                    layername = self.collections[mc].calc_bbfile_priority(appendfn)[2]
+                    ignore = self.databuilder.mcdata[mc].getVar("BB_DANGLINGAPPENDS_IGNORE_%s" % layername,
+                        False) or "no"
+                    if ignore.lower() not in ("1", "yes", "true"):
+                        appends_without_recipes[mc].append(appendfn)
 
         msgs = []
         for mc in sorted(appends_without_recipes.keys()):