diff mbox series

sstate: Fix sstate mirrors with BB_FETCH_PREMIRRORONLY

Message ID 20250210101252.219539-1-jhaller.oss@gmail.com
State New
Headers show
Series sstate: Fix sstate mirrors with BB_FETCH_PREMIRRORONLY | expand

Commit Message

Julian Haller Feb. 10, 2025, 10:12 a.m. UTC
From: Julian Haller <julian.haller@philips.com>

In case BB_FETCH_PREMIRRORONLY is set, the fetcher.download() function
sets BB_NO_NETWORK = "1" after downloading the first file. This alters
the precondition for the second and third loop run, disallowing network
access entirely. As a consequence, the .siginfo and .sig files can
never be downloaded.

Fix this by creating a new copy of localdata for each loop run.

Alternative solutions would be:
- Setting BB_FETCH_PREMIRRORONLY = "0" in localdata
- Setting BB_NO_NETWORK = "0" in each loop run

Both alternatives are highly dependent on the concrete implementation
of the fetcher function, thus making them likely to break in the
future.

Signed-off-by: Julian Haller <julian.haller@philips.com>
---
 meta/classes-global/sstate.bbclass | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

Comments

Richard Purdie Feb. 20, 2025, 10:25 p.m. UTC | #1
On Mon, 2025-02-10 at 11:12 +0100, Julian Haller via lists.openembedded.org wrote:
> From: Julian Haller <julian.haller@philips.com>
> 
> In case BB_FETCH_PREMIRRORONLY is set, the fetcher.download() function
> sets BB_NO_NETWORK = "1" after downloading the first file. This alters
> the precondition for the second and third loop run, disallowing network
> access entirely. As a consequence, the .siginfo and .sig files can
> never be downloaded.
> 
> Fix this by creating a new copy of localdata for each loop run.
> 
> Alternative solutions would be:
> - Setting BB_FETCH_PREMIRRORONLY = "0" in localdata
> - Setting BB_NO_NETWORK = "0" in each loop run
> 
> Both alternatives are highly dependent on the concrete implementation
> of the fetcher function, thus making them likely to break in the
> future.
> 
> Signed-off-by: Julian Haller <julian.haller@philips.com>
> ---
>  meta/classes-global/sstate.bbclass | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)

Thanks for the patch and raising this.

Having looked at the code, I think that rather than working around it,
we need to fix this in bitbake properly. I've copied you on such a
patch, testing/review would be appreciated.

Thanks,

Richard
diff mbox series

Patch

diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass
index 2c259a6657..eb988b2ca8 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -747,10 +747,11 @@  def pstaging_fetch(sstatefetch, d):
         uris += ['file://{0}.sig;downloadfilename={0}.sig'.format(sstatefetch)]
 
     for srcuri in uris:
-        localdata.delVar('SRC_URI')
-        localdata.setVar('SRC_URI', srcuri)
+        localdata2 = bb.data.createCopy(localdata)
+        localdata2.delVar('SRC_URI')
+        localdata2.setVar('SRC_URI', srcuri)
         try:
-            fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
+            fetcher = bb.fetch2.Fetch([srcuri], localdata2, cache=False)
             fetcher.checkstatus()
             fetcher.download()