| Message ID | 20260227072110.2807435-1-hongxu.jia@windriver.com |
|---|---|
| State | Under Review |
| Headers | show |
| Series | classes/yocto-check-layer: allow to explicitly skip check_network_flag in recipe | expand |
Hi hongxu, hongxu via lists.openembedded.org <hongxu.jia= windriver.com@lists.openembedded.org> escreveu (sexta, 27/02/2026 à(s) 07:21): > The yocto-check-layer add a new test that checks that no tasks > between do_fetch (exclusive) and do_build (inclusive) are allowed > to use the network, with rare exceptions. > > The only exception currently is build-appliance-image's do_image task, > which is hardcoded in yocto-check-layer bbclass. > > This commit adds variable SKIP_CHECK_NETWORK_FLAG to allow user to > explicitly skip check_network_flag in recipe > > Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> > --- > meta/classes-global/yocto-check-layer.bbclass | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/meta/classes-global/yocto-check-layer.bbclass > b/meta/classes-global/yocto-check-layer.bbclass > index ba93085325..9fb36f5646 100644 > --- a/meta/classes-global/yocto-check-layer.bbclass > +++ b/meta/classes-global/yocto-check-layer.bbclass > @@ -27,13 +27,16 @@ def check_insane_skip(d): > d.setVar("QA_ERRORS_FOUND", "True") > > > +# Format: "BPN1:task1 BPN2:task2", separate by space > +# build-appliance-image uses pip at image time > +SKIP_CHECK_NETWORK_FLAG = "build-appliance-image:do_image" > + > # Check that no tasks (with rare exceptions) between do_fetch and do_build > # use the network. > def check_network_flag(d): > # BPN:task names that are allowed to reach the network, using fnmatch > to compare. > allowed = [] > - # build-appliance-image uses pip at image time > - allowed += ["build-appliance-image:do_image"] > + allowed += (d.getVar('SKIP_CHECK_NETWORK_FLAG') or '').split() > This could introduce severe reproducibility problems for someone who claims to have a Yocto compatible layer. Can you tell what other recipes require this? Jose > > def is_allowed(bpn, task): > from fnmatch import fnmatch > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#232081): > https://lists.openembedded.org/g/openembedded-core/message/232081 > Mute This Topic: https://lists.openembedded.org/mt/118026872/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On 2/27/26 17:39, Jose Quaresma wrote: > ** > *CAUTION: This email comes from a non Wind River email account!* > Do not click links or open attachments unless you recognize the sender > and know the content is safe. > Hi hongxu, > > hongxu via lists.openembedded.org > <https://urldefense.com/v3/__http://lists.openembedded.org__;!!AjveYdw8EvQ!aBhDhydW4sVwHmPku-G3KfAkizU2zIqypxgoEenL-xjXJs5eoMzW0QXn8MVS7w9-QZvBeU26B0ju3x5wCHfoAWuj9pQ$> > <hongxu.jia=windriver.com@lists.openembedded.org> escreveu (sexta, > 27/02/2026 à(s) 07:21): > > The yocto-check-layer add a new test that checks that no tasks > between do_fetch (exclusive) and do_build (inclusive) are allowed > to use the network, with rare exceptions. > > The only exception currently is build-appliance-image's do_image task, > which is hardcoded in yocto-check-layer bbclass. > > This commit adds variable SKIP_CHECK_NETWORK_FLAG to allow user to > explicitly skip check_network_flag in recipe > > Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> > --- > meta/classes-global/yocto-check-layer.bbclass | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/meta/classes-global/yocto-check-layer.bbclass > b/meta/classes-global/yocto-check-layer.bbclass > index ba93085325..9fb36f5646 100644 > --- a/meta/classes-global/yocto-check-layer.bbclass > +++ b/meta/classes-global/yocto-check-layer.bbclass > @@ -27,13 +27,16 @@ def check_insane_skip(d): > d.setVar("QA_ERRORS_FOUND", "True") > > > +# Format: "BPN1:task1 BPN2:task2", separate by space > +# build-appliance-image uses pip at image time > +SKIP_CHECK_NETWORK_FLAG = "build-appliance-image:do_image" > + > # Check that no tasks (with rare exceptions) between do_fetch and > do_build > # use the network. > def check_network_flag(d): > # BPN:task names that are allowed to reach the network, using > fnmatch to compare. > allowed = [] > - # build-appliance-image uses pip at image time > - allowed += ["build-appliance-image:do_image"] > + allowed += (d.getVar('SKIP_CHECK_NETWORK_FLAG') or '').split() > > > This could introduce severe reproducibility problems for someone who > claims to have a Yocto compatible layer. > The meta-tensorflow, who use bazel build system to build, it requires network access at do_compile if download mirror is not available. The bazel is similar bitbake, has fetch, configure, compile, but it combined as one command and invoked at bitbake's do_compile In order to support offline build, I've apply a local patch to bazel to save download tarball as download mirror [1] [1] https://git.yoctoproject.org/meta-tensorflow/commit/?id=88ca1af3768e5a01e6ba8b2f09d6cf2a0bfb621e If dowload mirror is available, the build will reuse it and network is not required, the reproducibility problems should be detected by binary comparison from two builds, we have oe-selftest case in oe-core by the way //Hongxu > Can you tell what other recipes require this? > > Jose > > > def is_allowed(bpn, task): > from fnmatch import fnmatch > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#232081): > https://lists.openembedded.org/g/openembedded-core/message/232081 > <https://urldefense.com/v3/__https://lists.openembedded.org/g/openembedded-core/message/232081__;!!AjveYdw8EvQ!aBhDhydW4sVwHmPku-G3KfAkizU2zIqypxgoEenL-xjXJs5eoMzW0QXn8MVS7w9-QZvBeU26B0ju3x5wCHfov5GcSzk$> > Mute This Topic: > https://lists.openembedded.org/mt/118026872/5052612 > <https://urldefense.com/v3/__https://lists.openembedded.org/mt/118026872/5052612__;!!AjveYdw8EvQ!aBhDhydW4sVwHmPku-G3KfAkizU2zIqypxgoEenL-xjXJs5eoMzW0QXn8MVS7w9-QZvBeU26B0ju3x5wCHfodBBW8kA$> > Group Owner: openembedded-core+owner@lists.openembedded.org > <mailto:openembedded-core%2Bowner@lists.openembedded.org> > Unsubscribe: > https://lists.openembedded.org/g/openembedded-core/unsub > <https://urldefense.com/v3/__https://lists.openembedded.org/g/openembedded-core/unsub__;!!AjveYdw8EvQ!aBhDhydW4sVwHmPku-G3KfAkizU2zIqypxgoEenL-xjXJs5eoMzW0QXn8MVS7w9-QZvBeU26B0ju3x5wCHfo2v8OWTY$> > [quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > > > > -- > Best regards, > > José Quaresma
On Sat, 2026-02-28 at 11:27 +0800, hongxu via lists.openembedded.org wrote: > > On 2/27/26 17:39, Jose Quaresma wrote: > hongxu via lists.openembedded.org [1] > <hongxu.jia=windriver.com@lists.openembedded.org> escreveu (sexta, > 27/02/2026 à(s) 07:21): > > > > > > +# Format: "BPN1:task1 BPN2:task2", separate by space > > > +# build-appliance-image uses pip at image time > > > +SKIP_CHECK_NETWORK_FLAG = "build-appliance-image:do_image" > > > + > > > # Check that no tasks (with rare exceptions) between do_fetch > > > and do_build > > > # use the network. > > > def check_network_flag(d): > > > # BPN:task names that are allowed to reach the network, > > > using fnmatch to compare. > > > allowed = [] > > > - # build-appliance-image uses pip at image time > > > - allowed += ["build-appliance-image:do_image"] > > > + allowed += (d.getVar('SKIP_CHECK_NETWORK_FLAG') or > > > '').split() > > > > > > > > > > > > > This could introduce severe reproducibility problems for someone > > who claims to have a Yocto compatible layer. > > > > > > > > > > > > > > > > The meta-tensorflow, who use bazel build system to build, it requires > network access at do_compile if download mirror is not available. > > The bazel is similar bitbake, has fetch, configure, compile, but it > combined as one command and invoked at bitbake's do_compile > > In order to support offline build, I've apply a local patch to bazel > to save download tarball as download mirror [1] > > [1] > https://git.yoctoproject.org/meta-tensorflow/commit/?id=88ca1af3768e5a01e6ba8b2f09d6cf2a0bfb621e > > If dowload mirror is available, the build will reuse it and network > is not required, the reproducibility problems should be detected by > binary comparison from two builds, we have oe-selftest case in oe- > core by the way If the fetching happens outside of do_fetch, it means meta-tensorflow cannot be marked as Yocto Project Compatible. The point of the standard and this test is to move people towards reproducbile builds with full manifests of the contents. If you bypass the fetcher, we don't have any of these guarantees. Our plan was to work out a way to remove the fetching from build- appliance too but we didn't want to hold off the implementation of that on the rest of the standard. The fact we've not done that yet is frustrating to me but it doesn't change what the intent of this plan is. We don't want to add a way to bypass it unless there is really good reason. Good reasons might be 'publishing tasks' where we're writing data out to a remote, or we're running tests. I'd likely suggests these be in specific well defined tasks similar to fetch with known properties though. Cheers, Richard > [1] lists.openembedded.org https://urldefense.com/v3/__http://lists.openembedded.org__;!!AjveYdw8EvQ!aBhDhydW4sVwHmPku-G3KfAkizU2zIqypxgoEenL-xjXJs5eoMzW0QXn8MVS7w9-QZvBeU26B0ju3x5wCHfoAWuj9pQ$
diff --git a/meta/classes-global/yocto-check-layer.bbclass b/meta/classes-global/yocto-check-layer.bbclass index ba93085325..9fb36f5646 100644 --- a/meta/classes-global/yocto-check-layer.bbclass +++ b/meta/classes-global/yocto-check-layer.bbclass @@ -27,13 +27,16 @@ def check_insane_skip(d): d.setVar("QA_ERRORS_FOUND", "True") +# Format: "BPN1:task1 BPN2:task2", separate by space +# build-appliance-image uses pip at image time +SKIP_CHECK_NETWORK_FLAG = "build-appliance-image:do_image" + # Check that no tasks (with rare exceptions) between do_fetch and do_build # use the network. def check_network_flag(d): # BPN:task names that are allowed to reach the network, using fnmatch to compare. allowed = [] - # build-appliance-image uses pip at image time - allowed += ["build-appliance-image:do_image"] + allowed += (d.getVar('SKIP_CHECK_NETWORK_FLAG') or '').split() def is_allowed(bpn, task): from fnmatch import fnmatch
The yocto-check-layer add a new test that checks that no tasks between do_fetch (exclusive) and do_build (inclusive) are allowed to use the network, with rare exceptions. The only exception currently is build-appliance-image's do_image task, which is hardcoded in yocto-check-layer bbclass. This commit adds variable SKIP_CHECK_NETWORK_FLAG to allow user to explicitly skip check_network_flag in recipe Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> --- meta/classes-global/yocto-check-layer.bbclass | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)