diff mbox series

[4/5] bitbake-setup: correct several scenarios in layer updates

Message ID 20260109132000.2372791-4-alex.kanavin@gmail.com
State New
Headers show
Series [1/5] doc: document fixed revisions override in bitbake-setup manual | expand

Commit Message

Alexander Kanavin Jan. 9, 2026, 1:19 p.m. UTC 
From: Alexander Kanavin <alex@linutronix.de>

There were untested scenarios in layer updates which did not work properly.

The most serious of them is updating a remote layer which had user-made local
modifications (e.g. newly made local commits or edited files
without a commit). Bitbake-setup relies on git fetcher's unpack operation
which simply wipes it all out. Yes, the user should've used -L option
to symlink a local clone from elsewhere; this advice doesn't help when their
work is gone.

To address this, a git hook is added that prevents making commits in checkouts
under layers/ that are managed by bitbake-setup. The hook directs users to clone
the layer they'd like to modify separately and then use 'init -L' option.

(note: later on this workflow can be streamlined by converting parts of an existing
setup to local sources, or perhaps ability to 'detach' parts of layers/ from
bitbake-setup's modifications, so that it's not necessary to do a full re-init).

This still does not cover local modifications without making commits, so there's
also a function that checks if such modifications have been
made, and a code that moves the layer into a backup location if the function
returns true. I considered asking the user what to do (similar to bitbake
config handling), but this would've resulted in a half-updated, possibly
broken layer set which ultimately just would be more confusing.

The local modification check is a part of a broader newly added logic block
that handles already existing sources, and in particular addresses yet another
issue: when a local source replaces a git remote or another local source, this
scenario failed with 'file exists' errors (thanks to Anibal Limon for spotting
that). In this case the original symlink is removed first. If the original is a
git checkout, it is backed up if there are local modifications, or simply removed
otherwise.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 bin/bitbake-setup | 47 ++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 44 insertions(+), 3 deletions(-)
diff mbox series

Patch

diff --git a/bin/bitbake-setup b/bin/bitbake-setup
index 93c11bd09..abe7614c8 100755
--- a/bin/bitbake-setup
+++ b/bin/bitbake-setup
@@ -104,7 +104,7 @@  def add_unique_timestamp_to_path(path):
                 break
     return path_unique
 
-def checkout_layers(layers, layerdir, d):
+def checkout_layers(layers, confdir, layerdir, d):
     def _checkout_git_remote(r_remote, repodir, layers_fixed_revisions):
         rev = r_remote['rev']
         branch = r_remote.get('branch', None)
@@ -128,6 +128,31 @@  def checkout_layers(layers, layerdir, d):
         logger.plain("Making a symbolic link {} pointing to {}".format(dst, src))
         os.symlink(src, dst)
 
+    def _has_local_modifications(r_name, r_path):
+        fixed_revisions = json.load(open(os.path.join(confdir, "sources-fixed-revisions.json")))
+        rev = fixed_revisions['sources'][r_name]['git-remote']['rev']
+        status = bb.process.run('git -C {} status --porcelain'.format(r_path))[0]
+        if status:
+            return True
+        diff = bb.process.run('git -C {} diff {}'.format(r_path, rev))[0]
+        if diff:
+            return True
+        return False
+
+    def _restrict_commits(r_name, r_path):
+        hook_path = os.path.join(r_path, '.git', 'hooks', 'pre-commit')
+        restrict_hook = """#!/bin/sh
+echo "This repository is managed by bitbake-setup, and making commits is restricted.
+If you wish to make local modifications, clone it separately, and re-initialize using
+bitbake-setup init -L {} /path/to/repo/checkout"
+exit 1
+""".format(r_name)
+        with open(hook_path, 'w') as f:
+            f.write(restrict_hook)
+        import stat
+        st = os.stat(hook_path)
+        os.chmod(hook_path, st.st_mode | stat.S_IEXEC)
+
     layers_fixed_revisions = copy.deepcopy(layers)
     repodirs = []
     oesetupbuild = None
@@ -141,10 +166,26 @@  def checkout_layers(layers, layerdir, d):
         r_local = r_data.get('local')
         if r_remote and r_local:
             raise Exception("Source {} contains both git-remote and local properties.".format(r_name))
+
+        repodir_path = os.path.join(layerdir, repodir)
+        if os.path.lexists(repodir_path):
+            if os.path.islink(repodir_path):
+                os.remove(repodir_path)
+            elif _has_local_modifications(r_name, repodir_path):
+                backup_path = add_unique_timestamp_to_path(repodir_path + '-backup')
+                logger.warning("""Source {} in {} contains local modifications. Renaming to {} to preserve them.
+For local development work it is recommended to clone the needed layers separately and re-initialize using -L option:
+bitbake-setup init -L {} /path/to/repo/checkout""".format(
+                    r_name, repodir_path, backup_path, r_name))
+                os.rename(repodir_path, backup_path)
+            else:
+                shutil.rmtree(repodir_path)
+
         if r_remote:
             _checkout_git_remote(r_remote, repodir, layers_fixed_revisions)
+            _restrict_commits(r_name, repodir_path)
         if r_local:
-            _symlink_local(os.path.expanduser(r_local["path"]), os.path.join(layerdir,repodir))
+            _symlink_local(os.path.expanduser(r_local["path"]), repodir_path)
 
         if os.path.exists(os.path.join(layerdir, repodir, 'scripts/oe-setup-build')):
             oesetupbuild = os.path.join(layerdir, repodir, 'scripts/oe-setup-build')
@@ -354,7 +395,7 @@  def merge_overrides_into_sources(sources, overrides):
 
 def update_build(config, confdir, setupdir, layerdir, d, update_bb_conf="prompt"):
     layer_config = merge_overrides_into_sources(config["data"]["sources"], config["source-overrides"]["sources"])
-    sources_fixed_revisions = checkout_layers(layer_config, layerdir, d)
+    sources_fixed_revisions = checkout_layers(layer_config, confdir, layerdir, d)
     bitbake_config = config["bitbake-config"]
     thisdir = os.path.dirname(config["path"]) if config["type"] == 'local' else None
     setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_conf)