| Message ID | 20221207124913.2108579-1-richard.purdie@linuxfoundation.org |
|---|---|
| State | Accepted, archived |
| Commit | 6709aedccbb2e7ddbb1b2e7e4893481a7b536436 |
| Headers | show |
| Series | cookerdata: Ensure layers use LAYERSERIES_COMPAT fairly | expand |
Hi Richard, Am Mittwoch, dem 07.12.2022 um 12:49 +0000 schrieb Richard Purdie: > > Some layers think they're going to be 'clever' and copy the values from as I am one of those "clever" guys... > > another layer, e.g. using ${LAYERSERIES_COMPAT_core}. The whole point of > > this mechanism is to make it clear which releases a layer supports and FYI we "disabled" the mechanism because its not helping us. Support for us means commit level verification for hw+sw, so we had to implement something else anyway. > > show when a layer master branch is bitrotting and is unmaintained. I understand the argument especially for master > > > > Therefore add some code to avoid people doing this. I wish we didn't have > > to but... I just noticed in the documentation the requirement for layer compatible 2.0, enforcing that mechanism. I will fix it asap. As, I guess, there are more cases like that in our layer, for the future, instead of enforcing rules on the code level, it would help us more to make the yocto compatibility layer program more accessible. I tried several times on the management level to get into the program, without success. Our company did not fit into the existing categories. If you have an idea in that direction, a PM would be appreciated. Regards, Stefan > > > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > > --- > > lib/bb/cookerdata.py | 29 +++++++++++++++++++++++++++-- > > 1 file changed, 27 insertions(+), 2 deletions(-) > > > > diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py > > index 28cb59d83a..860968f2e8 100644 > > --- a/lib/bb/cookerdata.py > > +++ b/lib/bb/cookerdata.py > > @@ -383,6 +383,8 @@ class CookerDataBuilder(object): > > parselog.critical("Please check BBLAYERS in %s" % (layerconf)) > > raise bb.BBHandledException() > > > > + layerseries = None > > + compat_entries = {} > > for layer in layers: > > parselog.debug2("Adding layer %s", layer) > > if 'HOME' in approved and '~' in layer: > > @@ -395,6 +397,23 @@ class CookerDataBuilder(object): > > data.expandVarref('LAYERDIR') > > data.expandVarref('LAYERDIR_RE') > > > > + # Sadly we can't have nice things. > > + # Some layers think they're going to be 'clever' and copy the values from > > + # another layer, e.g. using ${LAYERSERIES_COMPAT_core}. The whole point of > > + # this mechanism is to make it clear which releases a layer supports and > > + # show when a layer master branch is bitrotting and is unmaintained. > > + # We therefore avoid people doing this here. > > + collections = (data.getVar('BBFILE_COLLECTIONS') or "").split() > > + for c in collections: > > + compat_entry = data.getVar("LAYERSERIES_COMPAT_%s" % c) > > + if compat_entry: > > + compat_entries[c] = set(compat_entry.split()) > > + data.delVar("LAYERSERIES_COMPAT_%s" % c) > > + if not layerseries: > > + layerseries = set((data.getVar("LAYERSERIES_CORENAMES") or "").split()) > > + if layerseries: > > + data.delVar("LAYERSERIES_CORENAMES") > > + > > data.delVar('LAYERDIR_RE') > > data.delVar('LAYERDIR') > > > > @@ -415,13 +434,17 @@ class CookerDataBuilder(object): > > if invalid: > > bb.fatal("BBFILES_DYNAMIC entries must be of the form {!}<collection name>:<filename > > pattern>, not:\n %s" % "\n ".join(invalid)) > > > > - layerseries = set((data.getVar("LAYERSERIES_CORENAMES") or "").split()) > > collections_tmp = collections[:] > > for c in collections: > > collections_tmp.remove(c) > > if c in collections_tmp: > > bb.fatal("Found duplicated BBFILE_COLLECTIONS '%s', check bblayers.conf or layer.conf > > to fix it." % c) > > - compat = set((data.getVar("LAYERSERIES_COMPAT_%s" % c) or "").split()) > > + > > + compat = set() > > + if c in compat_entries: > > + compat = compat_entries[c] > > + if compat: > > + data.delVar("LAYERSERIES_COMPAT_%s" % c) > > if compat and not layerseries: > > bb.fatal("No core layer found to work with layer '%s'. Missing entry in > > bblayers.conf?" > > % c) > > if compat and not (compat & layerseries): > > @@ -430,6 +453,8 @@ class CookerDataBuilder(object): > > elif not compat and not data.getVar("BB_WORKERCONTEXT"): > > bb.warn("Layer %s should set LAYERSERIES_COMPAT_%s in its conf/layer.conf file to > > list > > the core layer names it is compatible with." % (c, c)) > > > > + data.setVar("LAYERSERIES_CORENAMES", " ".join(layerseries)) > > + > > if not data.getVar("BBPATH"): > > msg = "The BBPATH variable is not set" > > if not layerconf: > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#14141): https://lists.openembedded.org/g/bitbake-devel/message/14141 > > Mute This Topic: https://lists.openembedded.org/mt/95514429/3617159 > > Group Owner: bitbake-devel+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [S.Mueller-Klieser@phytec.de] > > -=-=-=-=-=-=-=-=-=-=-=- > >
Hi Stefan, On Fri, 2022-12-09 at 11:07 +0000, Stefan Müller-Klieser wrote: > Am Mittwoch, dem 07.12.2022 um 12:49 +0000 schrieb Richard Purdie: > > > Some layers think they're going to be 'clever' and copy the values from > > as I am one of those "clever" guys... One of them, yes. You're not the only ones and the widespread adoption of this was a big part of the concern. > > > > another layer, e.g. using ${LAYERSERIES_COMPAT_core}. The whole point of > > > this mechanism is to make it clear which releases a layer supports and > > FYI we "disabled" the mechanism because its not helping us. Support for us > means commit level verification for hw+sw, so we had to implement something > else anyway. What you did was remove the indications of which releases your layer/branch was compatible with. I understand you may want to do other things as well but bypassing the checks really isn't in the spirit of what we were trying to do. > > > show when a layer master branch is bitrotting and is unmaintained. > > I understand the argument especially for master There are a lot of layers out there which it is unclear what they're compatible with and I do think having layers clearly indicate compatibility (or not) with a given release series is a good thing. > > > Therefore add some code to avoid people doing this. I wish we didn't have > > > to but... > > I just noticed in the documentation the requirement for layer > compatible 2.0, enforcing that mechanism. I will fix it asap. > > As, I guess, there are more cases like that in our layer, for > the future, instead of enforcing rules on the code level, it would > help us more to make the yocto compatibility layer program more > accessible. I tried several times on the management level to > get into the program, without success. Our company did > not fit into the existing categories. If you have an idea in that > direction, a PM would be appreciated. To use the Yocto Project Compatible branding/badge you need to be a member however you do not need to be a member to run the yocto-check- layer script yourself. You can say that the script passes, or to run it in automated CI and publish the result without any membership requirement. Is there something else we should be doing to make this more accessible? I'd be interested in what options could make membership more attractive to your management if you have any ideas. Cheers, Richard
Am Freitag, dem 09.12.2022 um 11:35 +0000 schrieb Richard Purdie: > Hi Stefan, > > On Fri, 2022-12-09 at 11:07 +0000, Stefan Müller-Klieser wrote: > > Am Mittwoch, dem 07.12.2022 um 12:49 +0000 schrieb Richard Purdie: > > > > Some layers think they're going to be 'clever' and copy the values from > > > > as I am one of those "clever" guys... > > One of them, yes. You're not the only ones and the widespread adoption > of this was a big part of the concern. > > > > > > > another layer, e.g. using ${LAYERSERIES_COMPAT_core}. The whole point of > > > > this mechanism is to make it clear which releases a layer supports and > > > > FYI we "disabled" the mechanism because its not helping us. Support for us > > means commit level verification for hw+sw, so we had to implement something > > else anyway. > > What you did was remove the indications of which releases your > layer/branch was compatible with. I understand you may want to do other > things as well but bypassing the checks really isn't in the spirit of > what we were trying to do. > > > > > show when a layer master branch is bitrotting and is unmaintained. > > > > I understand the argument especially for master > > There are a lot of layers out there which it is unclear what they're > compatible with and I do think having layers clearly indicate > compatibility (or not) with a given release series is a good thing. > > > > > Therefore add some code to avoid people doing this. I wish we didn't have > > > > to but... > > > > I just noticed in the documentation the requirement for layer > > compatible 2.0, enforcing that mechanism. I will fix it asap. > > > > As, I guess, there are more cases like that in our layer, for > > the future, instead of enforcing rules on the code level, it would > > help us more to make the yocto compatibility layer program more > > accessible. I tried several times on the management level to > > get into the program, without success. Our company did > > not fit into the existing categories. If you have an idea in that > > direction, a PM would be appreciated. > > To use the Yocto Project Compatible branding/badge you need to be a > member however you do not need to be a member to run the yocto-check- > layer script yourself. You can say that the script passes, or to run it > in automated CI and publish the result without any membership > requirement. > > Is there something else we should be doing to make this more > accessible? I'd be interested in what options could make membership > more attractive to your management if you have any ideas. Thanks, Richard. I just looked up the rules again. Our issue is that we are too small for a LF+Yocto Membership, not playing in that league. The Yocto Participant program would be a perfect fit, but its limited to 80 employees for a company. We are a vertically integrated company with 400 people, most working in manufacturing with a small development team focused on HW. So we fall in between the gap participant/silver member and are currently none. Something "official" and badges are really important to allocate resources in a company. There needs to be a communication channel to product management/ marketing so it does not end up being a hobby project for some developers. Any way, we try our best to keep up with upstream development. Thanks for you work and lets stay in touch. Regards, Stefan > > Cheers, > > Richard > >
diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py index 28cb59d83a..860968f2e8 100644 --- a/lib/bb/cookerdata.py +++ b/lib/bb/cookerdata.py @@ -383,6 +383,8 @@ class CookerDataBuilder(object): parselog.critical("Please check BBLAYERS in %s" % (layerconf)) raise bb.BBHandledException() + layerseries = None + compat_entries = {} for layer in layers: parselog.debug2("Adding layer %s", layer) if 'HOME' in approved and '~' in layer: @@ -395,6 +397,23 @@ class CookerDataBuilder(object): data.expandVarref('LAYERDIR') data.expandVarref('LAYERDIR_RE') + # Sadly we can't have nice things. + # Some layers think they're going to be 'clever' and copy the values from + # another layer, e.g. using ${LAYERSERIES_COMPAT_core}. The whole point of + # this mechanism is to make it clear which releases a layer supports and + # show when a layer master branch is bitrotting and is unmaintained. + # We therefore avoid people doing this here. + collections = (data.getVar('BBFILE_COLLECTIONS') or "").split() + for c in collections: + compat_entry = data.getVar("LAYERSERIES_COMPAT_%s" % c) + if compat_entry: + compat_entries[c] = set(compat_entry.split()) + data.delVar("LAYERSERIES_COMPAT_%s" % c) + if not layerseries: + layerseries = set((data.getVar("LAYERSERIES_CORENAMES") or "").split()) + if layerseries: + data.delVar("LAYERSERIES_CORENAMES") + data.delVar('LAYERDIR_RE') data.delVar('LAYERDIR') @@ -415,13 +434,17 @@ class CookerDataBuilder(object): if invalid: bb.fatal("BBFILES_DYNAMIC entries must be of the form {!}<collection name>:<filename pattern>, not:\n %s" % "\n ".join(invalid)) - layerseries = set((data.getVar("LAYERSERIES_CORENAMES") or "").split()) collections_tmp = collections[:] for c in collections: collections_tmp.remove(c) if c in collections_tmp: bb.fatal("Found duplicated BBFILE_COLLECTIONS '%s', check bblayers.conf or layer.conf to fix it." % c) - compat = set((data.getVar("LAYERSERIES_COMPAT_%s" % c) or "").split()) + + compat = set() + if c in compat_entries: + compat = compat_entries[c] + if compat: + data.delVar("LAYERSERIES_COMPAT_%s" % c) if compat and not layerseries: bb.fatal("No core layer found to work with layer '%s'. Missing entry in bblayers.conf?" % c) if compat and not (compat & layerseries): @@ -430,6 +453,8 @@ class CookerDataBuilder(object): elif not compat and not data.getVar("BB_WORKERCONTEXT"): bb.warn("Layer %s should set LAYERSERIES_COMPAT_%s in its conf/layer.conf file to list the core layer names it is compatible with." % (c, c)) + data.setVar("LAYERSERIES_CORENAMES", " ".join(layerseries)) + if not data.getVar("BBPATH"): msg = "The BBPATH variable is not set" if not layerconf:
Some layers think they're going to be 'clever' and copy the values from another layer, e.g. using ${LAYERSERIES_COMPAT_core}. The whole point of this mechanism is to make it clear which releases a layer supports and show when a layer master branch is bitrotting and is unmaintained. Therefore add some code to avoid people doing this. I wish we didn't have to but... Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- lib/bb/cookerdata.py | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)