| Message ID | 20250603085808.4045687-1-peter.suti@streamunlimited.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/1] icecc: calculate pn and bpn from FILE | expand |
On Tue, 2025-06-03 at 10:58 +0200, Peter Suti via lists.openembedded.org wrote: > Starting with Yocto Mickledore the PN and BPN variables are set to "no-pn" > at this point in the `use_icecc()` function, so instead we need to > re-parse them from the file again as a workaround otherwise icecc is broken. > > Fixes: 3be00ad9052de ("bitbake.conf: Add BB_HASH_CODEPARSER_VALS") > Where "PN=nopn" is set to optimise the codeparser cache's size. > > [0] https://lists.yoctoproject.org/g/yocto/topic/icecc_support_broken/103429714 > > Signed-off-by: Peter Suti <peter.suti@streamunlimited.com> > --- > meta/classes/icecc.bbclass | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass > index 8a48f2ad63..e3b07028b8 100644 > --- a/meta/classes/icecc.bbclass > +++ b/meta/classes/icecc.bbclass > @@ -143,8 +143,8 @@ def use_icecc(bb,d): > if icecc_is_cross_canadian(bb, d): > return "no" > > - pn = d.getVar('PN') > - bpn = d.getVar('BPN') > + pn = bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname' > + bpn = oe.utils.prune_suffix(pn, d.getVar('SPECIAL_PKGSUFFIX').split(), d) > > # Enable/disable checks are made against BPN, because there is a good > # chance that if icecc should be skipped for a recipe, it should be skipped Unfortunately PN could be set differently to that, or set through class extensions so I don't think this is a good way to fix the issue. It does highlight that icecc probably just worked through luck anyway. It would get it approximately right most of the time but I'm not sure we want to rely on that. To be honest, I'm more tempted to simply remove icecc since it would seem not a lot of people are using it if it takes this long for fixes to get sent :/. Cheers, Richard
On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via lists.openembedded.org wrote: > On Tue, 2025-06-03 at 10:58 +0200, Peter Suti via lists.openembedded.org wrote: > > Starting with Yocto Mickledore the PN and BPN variables are set to "no-pn" > > at this point in the `use_icecc()` function, so instead we need to > > re-parse them from the file again as a workaround otherwise icecc is broken. > > > > Fixes: 3be00ad9052de ("bitbake.conf: Add BB_HASH_CODEPARSER_VALS") > > Where "PN=nopn" is set to optimise the codeparser cache's size. > > > > [0] https://lists.yoctoproject.org/g/yocto/topic/icecc_support_broken/103429714 > > > > Signed-off-by: Peter Suti <peter.suti@streamunlimited.com> > > --- > > meta/classes/icecc.bbclass | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass > > index 8a48f2ad63..e3b07028b8 100644 > > --- a/meta/classes/icecc.bbclass > > +++ b/meta/classes/icecc.bbclass > > @@ -143,8 +143,8 @@ def use_icecc(bb,d): > > if icecc_is_cross_canadian(bb, d): > > return "no" > > > > - pn = d.getVar('PN') > > - bpn = d.getVar('BPN') > > + pn = bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname' > > + bpn = oe.utils.prune_suffix(pn, d.getVar('SPECIAL_PKGSUFFIX').split(), d) > > > > # Enable/disable checks are made against BPN, because there is a good > > # chance that if icecc should be skipped for a recipe, it should be skipped > > Unfortunately PN could be set differently to that, or set through class > extensions so I don't think this is a good way to fix the issue. It > does highlight that icecc probably just worked through luck anyway. It > would get it approximately right most of the time but I'm not sure we > want to rely on that. > > To be honest, I'm more tempted to simply remove icecc since it would > seem not a lot of people are using it if it takes this long for fixes > to get sent :/. Just to clarify, this would then let someone maintain this in a separate layer as it is self contained. I appreciate we'd need to find that maintainer someone needs to fill that role to keep this working... Cheers, Richard
On Tue, Jun 3, 2025 at 11:59 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via lists.openembedded.org wrote: > > Unfortunately PN could be set differently to that, or set through class > > extensions so I don't think this is a good way to fix the issue. It > > does highlight that icecc probably just worked through luck anyway. It > > would get it approximately right most of the time but I'm not sure we > > want to rely on that. Maybe that is why we have to disable a bunch of recipes with ICECC_RECIPE_DISABLE. > > To be honest, I'm more tempted to simply remove icecc since it would > > seem not a lot of people are using it if it takes this long for fixes > > to get sent :/. I understand. We will keep using it internally with the above patch because it works for our use cases and provides noticeable speedups still. > Just to clarify, this would then let someone maintain this in a > separate layer as it is self contained. I appreciate we'd need to find > that maintainer someone needs to fill that role to keep this working... > > Cheers, > > Richard Thank you for taking a look and the quick response! Best Regards, Peter
From: Peter Suti via lists.openembedded.org <peter.suti=streamunlimited.com@lists.openembedded.org> > On Tue, Jun 3, 2025 at 11:59 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > > > On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via lists.openembedded.org wrote: > > > Unfortunately PN could be set differently to that, or set through class > > > extensions so I don't think this is a good way to fix the issue. It > > > does highlight that icecc probably just worked through luck anyway. It > > > would get it approximately right most of the time but I'm not sure we > > > want to rely on that. > > Maybe that is why we have to disable a bunch of recipes with > ICECC_RECIPE_DISABLE. > > > > To be honest, I'm more tempted to simply remove icecc since it would > > > seem not a lot of people are using it if it takes this long for fixes > > > to get sent :/. > > I understand. We will keep using it internally with the above patch > because it works for our use cases and > provides noticeable speedups still. > > > Just to clarify, this would then let someone maintain this in a > > separate layer as it is self contained. I appreciate we'd need to find > > that maintainer someone needs to fill that role to keep this working... I suspect the reason no one has fixed this might be that no one else has hit it yet — we haven't. Fortunately in the meantime, Peter has found a work-around that works for him. I'd argue that icecc is well supported in oe-core. These are the top 6 contributors to icecc.bbclass, making 97 commits over 10 years, almost 10 per year: 21 Joshua Watt 18 Richard Purdie 14 Martin Jansa 11 Douglas Royds 9 Tobias Henkel 6 Dmitry Eremin-Solenikov I'm sure one of these contributors will bump into this same problem at some point, and that they'll fix it then. Please don't drop it from oe-core.
On Tue, 2025-06-03 at 21:55 +0000, Douglas Royds wrote: > From: Peter Suti via lists.openembedded.org > <peter.suti=streamunlimited.com@lists.openembedded.org> > > On Tue, Jun 3, 2025 at 11:59 AM Richard Purdie > > <richard.purdie@linuxfoundation.org> wrote: > > > > > > On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via > > > lists.openembedded.org wrote: > > > > Unfortunately PN could be set differently to that, or set > > > > through class > > > > extensions so I don't think this is a good way to fix the > > > > issue. It > > > > does highlight that icecc probably just worked through luck > > > > anyway. It > > > > would get it approximately right most of the time but I'm not > > > > sure we > > > > want to rely on that. > > > > Maybe that is why we have to disable a bunch of recipes with > > ICECC_RECIPE_DISABLE. > > > > > > To be honest, I'm more tempted to simply remove icecc since it > > > > would > > > > seem not a lot of people are using it if it takes this long for > > > > fixes > > > > to get sent :/. > > > > I understand. We will keep using it internally with the above patch > > because it works for our use cases and > > provides noticeable speedups still. > > > > > Just to clarify, this would then let someone maintain this in a > > > separate layer as it is self contained. I appreciate we'd need to > > > find > > > that maintainer someone needs to fill that role to keep this > > > working... > > I suspect the reason no one has fixed this might be that no one else > has hit it yet — we haven't. Fortunately in the meantime, Peter has > found a work-around that works for him. Which branches are you using this with? > I'd argue that icecc is well supported in oe-core. These are the top > 6 contributors to icecc.bbclass, making 97 commits over 10 years, > almost 10 per year: > > 21 Joshua Watt > 18 Richard Purdie > 14 Martin Jansa > 11 Douglas Royds > 9 Tobias Henkel > 6 Dmitry Eremin-Solenikov > > I'm sure one of these contributors will bump into this same problem > at some point, and that they'll fix it then. Please don't drop it > from oe-core. I can safely say I simply don't use it and never have, which hints at the kinds of commits I'm responsible for. Joshua used to but doesn't now and hasn't for a few years. As far as I know, Martin doesn't any more either. I'm therefore not sure this data makes a good case for it being well supported. At this point I think it belongs in its own layer. I appreciate people don't want to hear that and yes, that means someone will have to actually step up maintain it rather than relying on me and others somehow keeping it working mostly by luck. Cheers, Richard
On Tue, Jun 3, 2025 at 4:15 PM Richard Purdie via lists.openembedded.org <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > > On Tue, 2025-06-03 at 21:55 +0000, Douglas Royds wrote: > > From: Peter Suti via lists.openembedded.org > > <peter.suti=streamunlimited.com@lists.openembedded.org> > > > On Tue, Jun 3, 2025 at 11:59 AM Richard Purdie > > > <richard.purdie@linuxfoundation.org> wrote: > > > > > > > > On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via > > > > lists.openembedded.org wrote: > > > > > Unfortunately PN could be set differently to that, or set > > > > > through class > > > > > extensions so I don't think this is a good way to fix the > > > > > issue. It > > > > > does highlight that icecc probably just worked through luck > > > > > anyway. It > > > > > would get it approximately right most of the time but I'm not > > > > > sure we > > > > > want to rely on that. > > > > > > Maybe that is why we have to disable a bunch of recipes with > > > ICECC_RECIPE_DISABLE. > > > > > > > > To be honest, I'm more tempted to simply remove icecc since it > > > > > would > > > > > seem not a lot of people are using it if it takes this long for > > > > > fixes > > > > > to get sent :/. > > > > > > I understand. We will keep using it internally with the above patch > > > because it works for our use cases and > > > provides noticeable speedups still. > > > > > > > Just to clarify, this would then let someone maintain this in a > > > > separate layer as it is self contained. I appreciate we'd need to > > > > find > > > > that maintainer someone needs to fill that role to keep this > > > > working... > > > > I suspect the reason no one has fixed this might be that no one else > > has hit it yet — we haven't. Fortunately in the meantime, Peter has > > found a work-around that works for him. > > Which branches are you using this with? > > > I'd argue that icecc is well supported in oe-core. These are the top > > 6 contributors to icecc.bbclass, making 97 commits over 10 years, > > almost 10 per year: > > > > 21 Joshua Watt > > 18 Richard Purdie > > 14 Martin Jansa > > 11 Douglas Royds > > 9 Tobias Henkel > > 6 Dmitry Eremin-Solenikov > > > > I'm sure one of these contributors will bump into this same problem > > at some point, and that they'll fix it then. Please don't drop it > > from oe-core. > > I can safely say I simply don't use it and never have, which hints at > the kinds of commits I'm responsible for. Joshua used to but doesn't > now and hasn't for a few years. As far as I know, Martin doesn't any > more either. I'm therefore not sure this data makes a good case for it > being well supported. > > At this point I think it belongs in its own layer. I appreciate people > don't want to hear that and yes, that means someone will have to > actually step up maintain it rather than relying on me and others > somehow keeping it working mostly by luck. I agree. icecc can be quite useful, but it is one of those things that is basically impossible to test unless you are actively using it (e.g. it can't be easily tested on the autobuilder). I don't use it anymore, and I don't even have a setup to test it on, so I can't really provide much help on keeping it functional. > > Cheers, > > Richard > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#217828): https://lists.openembedded.org/g/openembedded-core/message/217828 > Mute This Topic: https://lists.openembedded.org/mt/113444145/3616693 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass index 8a48f2ad63..e3b07028b8 100644 --- a/meta/classes/icecc.bbclass +++ b/meta/classes/icecc.bbclass @@ -143,8 +143,8 @@ def use_icecc(bb,d): if icecc_is_cross_canadian(bb, d): return "no" - pn = d.getVar('PN') - bpn = d.getVar('BPN') + pn = bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname' + bpn = oe.utils.prune_suffix(pn, d.getVar('SPECIAL_PKGSUFFIX').split(), d) # Enable/disable checks are made against BPN, because there is a good # chance that if icecc should be skipped for a recipe, it should be skipped
Starting with Yocto Mickledore the PN and BPN variables are set to "no-pn" at this point in the `use_icecc()` function, so instead we need to re-parse them from the file again as a workaround otherwise icecc is broken. Fixes: 3be00ad9052de ("bitbake.conf: Add BB_HASH_CODEPARSER_VALS") Where "PN=nopn" is set to optimise the codeparser cache's size. [0] https://lists.yoctoproject.org/g/yocto/topic/icecc_support_broken/103429714 Signed-off-by: Peter Suti <peter.suti@streamunlimited.com> --- meta/classes/icecc.bbclass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)