diff mbox series

buildhistory.bbclass: restore BUILDHISTORY_PRESERVE files

Message ID 20250115153149.1827119-1-pmi183@gmail.com
State New
Headers show
Series buildhistory.bbclass: restore BUILDHISTORY_PRESERVE files | expand

Commit Message

Pedro Ferreira Jan. 15, 2025, 3:31 p.m. UTC
From: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com>

On each build using sstate-cache, buildhistory will move
content to a temporary folder named `old`.
When buildhistory looks for the main dir, it wont find it
and ends up creating it.
As a consequence how code is structured wont restore any
preserved file.

Code block moved to ensure if old dir exists, it will
attempt to restore those files marked to preserve.

Signed-off-by: Pedro Silva Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com>
---
 meta/classes/buildhistory.bbclass | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

Comments

Ross Burton Jan. 22, 2025, 1:42 p.m. UTC | #1
Hi Pedro,

> On 15 Jan 2025, at 15:31, Pedro Ferreira via lists.openembedded.org <pmi183=gmail.com@lists.openembedded.org> wrote:
> 
> From: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com>
> 
> On each build using sstate-cache, buildhistory will move
> content to a temporary folder named `old`.
> When buildhistory looks for the main dir, it wont find it
> and ends up creating it.
> As a consequence how code is structured wont restore any
> preserved file.
> 
> Code block moved to ensure if old dir exists, it will
> attempt to restore those files marked to preserve.

Do you have a reproducer for this behaviour?  As you have seen the buildhistory file management isn’t trivial and we’d like to understand the failure better.

Cheers,
Ross
Pedro Ferreira Jan. 23, 2025, 2:37 p.m. UTC | #2
Hi Ross,

Here's a simple test do reproduce it.

*Pre-conditions:*
INHERIT += "buildhistory"
BUILDHISTORY_RESET = "1"
BUILDHISTORY_PRESERVE:append = " files-in-package.txt"

*- Starting without sstate cache*
*Run:*
bitbake base-passwd
*Check:*
cat PATH_TO_BUILD_dir/buildhistory/packages/core2-64-poky-linux/base-passwd/base-passwd-dbg/files-in-package.txt
(File exists and content ok)
*Run:*
bitbake base-passwd (it will use sstate cache)
*Check:*
cat PATH_TO_BUILD_dir/buildhistory/packages/core2-64-poky-linux/base-passwd/base-passwd-dbg/files-in-package.txt
(File wont exist)

*- Starting with sstate cache*
*Run:*
bitbake base-passwd --no-setscene
*Check:*
cat PATH_TO_BUILD_dir/buildhistory/packages/core2-64-poky-linux/base-passwd/base-passwd-dbg/files-in-package.txt
(File exists and content ok)
*Run:*
bitbake base-passwd (it will use sstate cache)
*Check:*
cat PATH_TO_BUILD_dir/buildhistory/packages/core2-64-poky-linux/base-passwd/base-passwd-dbg/files-in-package.txt
(File wont exist)

Cheers,
Pedro
diff mbox series

Patch

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index d735dd5fb5..b0f395f05e 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -260,14 +260,6 @@  python buildhistory_emit_pkghistory() {
     if not os.path.exists(pkghistdir):
         bb.utils.mkdirhier(pkghistdir)
     else:
-        # We need to make sure that all files kept in
-        # buildhistory/old are restored successfully
-        # otherwise next block of code wont have files to
-        # check and purge
-        if d.getVar("BUILDHISTORY_RESET"):
-            for pkg in packagelist:
-                preservebuildhistoryfiles(pkg, preserve)
-
         # Remove files for packages that no longer exist
         for item in os.listdir(pkghistdir):
             if item not in preserve:
@@ -280,6 +272,13 @@  python buildhistory_emit_pkghistory() {
                     else:
                         os.unlink(itempath)
 
+    if os.path.exists(oldpkghistdir):
+        # We need to make sure that all files in preserve
+        # are restored from buildhistory/old successfully
+        if d.getVar("BUILDHISTORY_RESET"):
+            for pkg in packagelist:
+                preservebuildhistoryfiles(pkg, preserve)
+
     rcpinfo = RecipeInfo(pn)
     rcpinfo.pe = pe
     rcpinfo.pv = pv