From patchwork Sun Mar 22 19:34:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 260C7D58B00 for ; Sun, 22 Mar 2026 19:35:08 +0000 (UTC) Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net [185.136.65.228]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.1829.1774208105449353918 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=QqBN23xz; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.228, mailfrom: fm-1329275-20260322193502b98fb1678300020799-fs2sr0@rts-flowmailer.siemens.com) Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 20260322193502b98fb1678300020799 for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=rH8gvLjQh1iwN77TDQf3SGvr2xKl1fTyiMLwZGa/N6E=; b=QqBN23xz3tq4IeWTbmnMQB6Ad/5KU1CimZmut39q8sftgDTwAoMxIQINV/UWQSvS7jKEDV 8aUa7BX06m6/Rg0LVmkS5FWJ6FYvMSa5uKhcIC8diTWbpoBzPhLWeI60Ll/+3Yngb1NMxkEH CGVikoosPHQnJPTjE1PHvN8qhHa9b3Ea8onneLcegCKpB6IzjSdcAPxFRX/fTmgYXrm+DCON Gg6u9rhKiyJyYuJtf6MbN+CiXyVVFFk4Yt3+lSBZ+6f+sR6saGCl6cz15b+hNlX00wtwipjT p4AdXH6pz+i/18EYpLSGuAubHM+9mRl/6bmcN4JpWJwNvN9MUXzjtCGw==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 1/9] bitbake-selftest: add GitUnpackUpdateTest Date: Sun, 22 Mar 2026 20:34:12 +0100 Message-ID: <20260322193440.870120-2-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19205 From: Adrian Freihofer Add a test class that exercises the new unpack_update() code path in the Git fetcher, ordered from basic building blocks to advanced workflow and error cases: test_unpack_update_full_clone Basic update to a newer upstream revision succeeds and the working tree reflects the new content. test_unpack_update_dldir_remote_setup The "dldir" remote pointing to ud.clonedir is created during the initial unpack and is present for subsequent update calls. test_unpack_update_ff_with_local_changes Full workflow: after a normal unpack the "dldir" remote is verified, a local commit is added, download() brings updated_rev into the clonedir, and unpack_update() fetches from dldir and rebases the local commit fast forward on top. The commit graph (HEAD^ == updated_rev) and both file contents are asserted. test_unpack_update_already_at_target_revision Calling unpack_update() when the checkout is already at SRCREV is a no-op: it succeeds and the working tree is left unchanged. test_unpack_update_with_untracked_file The status check uses --untracked-files=no so untracked files are invisible to it; the update succeeds and the untracked file survives the rebase unchanged. test_unpack_update_with_staged_changes Staged (but not committed) changes cause git to refuse to rebase under --no-autostash; UnpackError is raised so the caller can fall back to backup + re-fetch. test_unpack_update_with_modified_tracked_file An unstaged modification to a tracked file is detected by "git status --untracked-files=no --porcelain" and blocks the update; UnpackError is raised. test_unpack_update_conflict_raises_unpack_error A local commit that conflicts with the incoming upstream change raises UnpackError; the repository is left in a clean state with no pending rebase (rebase --abort was called). test_unpack_update_untracked_file_overwritten_by_upstream An untracked file that would be overwritten by an incoming upstream commit causes git to refuse the rebase; UnpackError is raised and the repository is not left in a mid-rebase state. Two sub-cases are covered: a top-level file clash and a clash inside a subdirectory (xxx/somefile). test_unpack_update_shallow_clone_fails Shallow clones do not carry enough history; UnpackError is raised. test_unpack_update_stale_dldir_remote When the clonedir has been removed after the initial unpack the dldir remote no longer resolves; UnpackError is raised so the caller can fall back to a full re-fetch. test_fetch_unpack_update_toplevel_api The public Fetch.unpack_update(root) API (used by callers such as bitbake-setup) dispatches correctly end-to-end through to the Git fetcher. Signed-off-by: Adrian Freihofer --- lib/bb/tests/fetch.py | 563 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 559 insertions(+), 4 deletions(-) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 7b8297a78..b496bb4d3 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -18,7 +18,6 @@ import os import signal import tarfile from bb.fetch2 import URI -from bb.fetch2 import FetchMethod import bb import bb.utils from bb.tests.support.httpserver import HTTPService @@ -551,8 +550,8 @@ class MirrorUriTest(FetcherTest): fetcher = bb.fetch.FetchData("http://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz", self.d) mirrors = bb.fetch2.mirror_from_string(mirrorvar) uris, uds = bb.fetch2.build_mirroruris(fetcher, mirrors, self.d) - self.assertEqual(uris, ['file:///somepath/downloads/bitbake-1.0.tar.gz', - 'file:///someotherpath/downloads/bitbake-1.0.tar.gz', + self.assertEqual(uris, ['file:///somepath/downloads/bitbake-1.0.tar.gz', + 'file:///someotherpath/downloads/bitbake-1.0.tar.gz', 'http://otherdownloads.yoctoproject.org/downloads/bitbake-1.0.tar.gz', 'http://downloads2.yoctoproject.org/downloads/bitbake-1.0.tar.gz']) @@ -1390,7 +1389,7 @@ class URLHandle(unittest.TestCase): "https://somesite.com/somerepo.git;user=anyUser:idtoken=1234" : ('https', 'somesite.com', '/somerepo.git', '', '', {'user': 'anyUser:idtoken=1234'}), 'git://s.o-me_ONE:%s@git.openembedded.org/bitbake;branch=main;protocol=https' % password: ('git', 'git.openembedded.org', '/bitbake', 's.o-me_ONE', password, {'branch': 'main', 'protocol' : 'https'}), } - # we require a pathname to encodeurl but users can still pass such urls to + # we require a pathname to encodeurl but users can still pass such urls to # decodeurl and we need to handle them decodedata = datatable.copy() decodedata.update({ @@ -3793,3 +3792,559 @@ class GoModGitTest(FetcherTest): self.assertTrue(os.path.exists(os.path.join(downloaddir, 'go.opencensus.io/@v/v0.24.0.mod'))) self.assertEqual(bb.utils.sha256_file(os.path.join(downloaddir, 'go.opencensus.io/@v/v0.24.0.mod')), '0dc9ccc660ad21cebaffd548f2cc6efa27891c68b4fbc1f8a3893b00f1acec96') + + +class GitUnpackUpdateTest(FetcherTest): + """Test the unpack_update functionality for git fetcher. + + Intended workflow + 1. First-time setup: + 1. download() — clones the upstream repo into DL_DIR/git2/... (clonedir). + 2. unpack() — clones from clonedir into the workspace (S/workdir) and + registers a 'dldir' git remote pointing at + file://DL_DIR/git2/... for later offline use. + + 2. Subsequent updates (what unpack_update is designed for): + 1. The user works in the unpacked source tree. + 2. Upstream advances — SRCREV changes in the recipe. + 3. download() — fetches the new revision into the local clonedir. + 4. unpack_update() — instead of wiping the workspace and re-cloning: + * fetches the new revision from the local 'dldir' remote + * rebases the user's local commits on top of the new SRCREV + * raises UnpackError if anything prevents a clean rebase so the + caller (e.g. bitbake-setup) can fall back to backup + re-clone. + + Key design constraints: + * unpack_update() never deletes existing data (unlike unpack()). + * Only staged/modified tracked files block the update; untracked files and + committed local work are handled gracefully. + * The 'dldir' remote is intentionally visible to users outside the + fetcher (e.g. for manual 'git log dldir/master'). + * Currently only git is supported. + """ + + def setUp(self): + """Set up a local bare git source repository with two commits on 'master'. + + self.initial_rev — the first commit (testfile.txt: 'initial content') + self.updated_rev — the second commit (testfile.txt: 'updated content') + + SRCREV is initialised to self.initial_rev so individual tests can + advance it to self.updated_rev (or create further commits) as needed. + """ + FetcherTest.setUp(self) + + self.gitdir = os.path.join(self.tempdir, 'gitrepo') + self.srcdir = os.path.join(self.tempdir, 'gitsource') + + self.d.setVar('WORKDIR', self.tempdir) + self.d.setVar('S', self.gitdir) + self.d.delVar('PREMIRRORS') + self.d.delVar('MIRRORS') + + # Create a source git repository + bb.utils.mkdirhier(self.srcdir) + self.git_init(cwd=self.srcdir) + + # Create initial commit + with open(os.path.join(self.srcdir, 'testfile.txt'), 'w') as f: + f.write('initial content\n') + self.git(['add', 'testfile.txt'], cwd=self.srcdir) + self.git(['commit', '-m', 'Initial commit'], cwd=self.srcdir) + self.initial_rev = self.git(['rev-parse', 'HEAD'], cwd=self.srcdir).strip() + + # Create a second commit + with open(os.path.join(self.srcdir, 'testfile.txt'), 'w') as f: + f.write('updated content\n') + self.git(['add', 'testfile.txt'], cwd=self.srcdir) + self.git(['commit', '-m', 'Update commit'], cwd=self.srcdir) + self.updated_rev = self.git(['rev-parse', 'HEAD'], cwd=self.srcdir).strip() + + self.d.setVar('SRCREV', self.initial_rev) + self.d.setVar('SRC_URI', 'git://%s;branch=master;protocol=file' % self.srcdir) + + def test_unpack_update_full_clone(self): + """Test that unpack_update updates an existing checkout in place for a full clone. + + Steps: + 1. Fetch and unpack at self.initial_rev — verify 'initial content'. + 2. Advance SRCREV to self.updated_rev and re-download. + 3. Call unpack_update() instead of unpack() — the existing checkout + must be updated via 'git fetch dldir' + 'git rebase' without + re-cloning the directory. + 4. Verify testfile.txt now contains 'updated content'. + """ + # First fetch at initial revision + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + # Verify initial state + unpack_path = os.path.join(self.unpackdir, 'git') + self.assertTrue(os.path.exists(os.path.join(unpack_path, 'testfile.txt'))) + with open(os.path.join(unpack_path, 'testfile.txt'), 'r') as f: + self.assertEqual(f.read(), 'initial content\n') + + # Update to new revision + self.d.setVar('SRCREV', self.updated_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + # Use unpack_update + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + git_fetcher = ud.method + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + # Verify updated state + with open(os.path.join(unpack_path, 'testfile.txt'), 'r') as f: + self.assertEqual(f.read(), 'updated content\n') + + def test_unpack_update_dldir_remote_setup(self): + """Test that unpack() adds a 'dldir' git remote pointing at ud.clonedir. + + The 'dldir' remote is used by subsequent unpack_update() calls to fetch + new commits from the local download cache (${DL_DIR}/git2/…) without + requiring network access. After a normal unpack the remote must exist + and its URL must be 'file://'. + """ + # First fetch + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + + # Check that dldir remote exists + remotes = self.git(['remote'], cwd=unpack_path).strip().split('\n') + self.assertIn('dldir', remotes) + + # Verify it points to the clonedir + dldir_url = self.git(['remote', 'get-url', 'dldir'], cwd=unpack_path).strip() + self.assertEqual(dldir_url, 'file://{}'.format(ud.clonedir)) + + def test_unpack_update_ff_with_local_changes(self): + """Test that unpack_update rebases local commits fast forward. + + Full workflow: + 1. Fetch + unpack at initial_rev — verify 'dldir' remote is created + pointing at ud.clonedir. + 2. Add a local commit touching localfile.txt. + 3. Advance SRCREV to updated_rev and call download() — verify that + ud.clonedir (the dldir bare clone) now contains updated_rev. + 4. Call unpack_update() — it fetches updated_rev from dldir into the + working tree and rebases the local commit on top. + 5. Verify the final commit graph: HEAD's parent is updated_rev, and + both testfile.txt ('updated content') and localfile.txt ('local + change') are present. + + Note: git rebase operates the same way regardless of whether HEAD is + detached or on a named branch (e.g. 'master' or a local feature branch), + so this test covers those scenarios implicitly. + """ + # Step 1 — fetch + unpack at initial_rev + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + unpack_path = os.path.join(self.unpackdir, 'git') + + # The normal unpack must have set up the 'dldir' remote pointing at + # ud.clonedir so that subsequent unpack_update() calls work offline. + dldir_url = self.git(['remote', 'get-url', 'dldir'], cwd=unpack_path).strip() + self.assertEqual(dldir_url, 'file://{}'.format(ud.clonedir)) + + # Step 2 — add a local commit that touches a new file + with open(os.path.join(unpack_path, 'localfile.txt'), 'w') as f: + f.write('local change\n') + self.git(['add', 'localfile.txt'], cwd=unpack_path) + self.git(['commit', '-m', 'Local commit'], cwd=unpack_path) + local_commit = self.git(['rev-parse', 'HEAD'], cwd=unpack_path).strip() + + # Step 3 — advance SRCREV and download; clonedir must now contain + # updated_rev so that unpack_update can fetch it without network access. + self.d.setVar('SRCREV', self.updated_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + ud = fetcher.ud[uri] + clonedir_refs = self.git(['rev-parse', self.updated_rev], cwd=ud.clonedir).strip() + self.assertEqual(clonedir_refs, self.updated_rev, + "clonedir must contain updated_rev after download()") + + # Step 4 — unpack_update fetches from dldir and rebases + git_fetcher = ud.method + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + # Step 5 — verify the commit graph and working tree content + # HEAD is the rebased local commit; its parent must be updated_rev + head_rev = self.git(['rev-parse', 'HEAD'], cwd=unpack_path).strip() + parent_rev = self.git(['rev-parse', 'HEAD^'], cwd=unpack_path).strip() + self.assertNotEqual(head_rev, local_commit, + "local commit should have a new SHA after rebase") + self.assertEqual(parent_rev, self.updated_rev, + "HEAD's parent must be updated_rev after fast-forward rebase") + + with open(os.path.join(unpack_path, 'testfile.txt'), 'r') as f: + self.assertEqual(f.read(), 'updated content\n') + with open(os.path.join(unpack_path, 'localfile.txt'), 'r') as f: + self.assertEqual(f.read(), 'local change\n') + + def test_unpack_update_already_at_target_revision(self): + """Test that unpack_update is a no-op when the checkout is already at SRCREV. + + Calling unpack_update() without advancing SRCREV must succeed and leave + the working tree unchanged. No rebase should be attempted because the + checkout already points at ud.revision. + """ + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + with open(os.path.join(unpack_path, 'testfile.txt')) as f: + self.assertEqual(f.read(), 'initial content\n') + + # Call unpack_update with SRCREV still at initial_rev — no upstream change + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + git_fetcher = ud.method + result = git_fetcher.unpack_update(ud, self.unpackdir, self.d) + self.assertTrue(result) + + # Content must be unchanged + with open(os.path.join(unpack_path, 'testfile.txt')) as f: + self.assertEqual(f.read(), 'initial content\n') + + def test_unpack_update_with_untracked_file(self): + """Test that unpack_update succeeds when the checkout has an untracked file. + + The status check uses '--untracked-files=no', so untracked files are not + detected and do not trigger the fallback path. git rebase also leaves + untracked files untouched, so both the upstream update and the untracked + file must be present after the call. + """ + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + + # Create an untracked file (not staged, not committed) + untracked = os.path.join(unpack_path, 'untracked.txt') + with open(untracked, 'w') as f: + f.write('untracked content\n') + + # Update to new upstream revision + self.d.setVar('SRCREV', self.updated_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + git_fetcher = ud.method + + # --untracked-files=no means the status check passes; rebase preserves the file + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + with open(os.path.join(unpack_path, 'testfile.txt'), 'r') as f: + self.assertEqual(f.read(), 'updated content\n') + + # Untracked file must survive the rebase + self.assertTrue(os.path.exists(untracked)) + with open(untracked, 'r') as f: + self.assertEqual(f.read(), 'untracked content\n') + + def test_unpack_update_with_staged_changes(self): + """Test that unpack_update fails when the checkout has staged (but not committed) changes. + + The rebase is run with --no-autostash so git refuses to rebase over a + dirty index. The caller (bitbake-setup) is expected to catch the + resulting UnpackError and fall back to backup + re-fetch. + """ + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + + # Stage a new file without committing it + staged = os.path.join(unpack_path, 'staged.txt') + with open(staged, 'w') as f: + f.write('staged content\n') + self.git(['add', 'staged.txt'], cwd=unpack_path) + + # Update to new upstream revision + self.d.setVar('SRCREV', self.updated_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + git_fetcher = ud.method + + # Should fail — git rebase refuses to run with a dirty index + with self.assertRaises(bb.fetch2.UnpackError): + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + def test_unpack_update_with_modified_tracked_file(self): + """Test that unpack_update fails when a tracked file has unstaged modifications. + + 'git status --untracked-files=no --porcelain' reports unstaged modifications + to tracked files (output line ' M filename'), which must block the update so + the caller can fall back to backup + re-fetch rather than silently discarding + work in progress. + """ + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + + # Modify a tracked file without staging or committing + with open(os.path.join(unpack_path, 'testfile.txt'), 'w') as f: + f.write('locally modified content\n') + + # Update to new upstream revision + self.d.setVar('SRCREV', self.updated_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + git_fetcher = ud.method + + # Should fail — unstaged modification to tracked file is detected by + # 'git status --untracked-files=no --porcelain' + with self.assertRaises(bb.fetch2.UnpackError): + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + def test_unpack_update_conflict_raises_unpack_error(self): + """Test that unpack_update raises UnpackError on a rebase conflict. + + When a local commit modifies the same lines as an incoming upstream commit, + git rebase cannot resolve the conflict automatically. unpack_update must + abort the failed rebase and raise UnpackError so the caller can fall back + to a backup + re-fetch. + """ + # Fetch and unpack at the initial revision + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + + # Make a local commit that edits the same lines as the upcoming upstream commit + with open(os.path.join(unpack_path, 'testfile.txt'), 'w') as f: + f.write('conflicting local content\n') + self.git(['add', 'testfile.txt'], cwd=unpack_path) + self.git(['commit', '-m', 'Local conflicting commit'], cwd=unpack_path) + + # Add a third upstream commit that also edits testfile.txt differently + with open(os.path.join(self.srcdir, 'testfile.txt'), 'w') as f: + f.write('conflicting upstream content\n') + self.git(['add', 'testfile.txt'], cwd=self.srcdir) + self.git(['commit', '-m', 'Upstream conflicting commit'], cwd=self.srcdir) + conflict_rev = self.git(['rev-parse', 'HEAD'], cwd=self.srcdir).strip() + + # Update SRCREV to the new upstream commit + self.d.setVar('SRCREV', conflict_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + git_fetcher = ud.method + + # unpack_update must fail and clean up (rebase --abort) rather than + # leaving the repo in a mid-rebase state + with self.assertRaises(bb.fetch2.UnpackError): + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + # Verify the repo is not left in a conflicted / mid-rebase state + rebase_merge = os.path.join(unpack_path, '.git', 'rebase-merge') + rebase_apply = os.path.join(unpack_path, '.git', 'rebase-apply') + self.assertFalse(os.path.exists(rebase_merge), + "rebase-merge dir should not exist after failed unpack_update") + self.assertFalse(os.path.exists(rebase_apply), + "rebase-apply dir should not exist after failed unpack_update") + + def test_unpack_update_untracked_file_overwritten_by_upstream(self): + """Test that unpack_update raises UnpackError when an untracked file would be + overwritten by an incoming upstream commit. + + We skip untracked files in the pre-check (git rebase doesn't touch harmless + untracked files), but git itself refuses to rebase when an untracked file would + be overwritten by the incoming changes. The resulting FetchError must be caught + and re-raised as UnpackError without leaving the repo in a mid-rebase state. + + Two sub-cases are covered: + - top-level untracked file clashing with an incoming upstream file + - untracked file inside a subdirectory (xxx/somefile) clashing with an + upstream commit that adds the same path + """ + def _run_case(upstream_path, local_rel_path, commit_msg): + """ + Add upstream_path to self.srcdir, create local_rel_path as an + untracked file in the checkout, then assert that unpack_update + raises UnpackError and leaves no mid-rebase state, and that the + local file is untouched. + """ + # Fresh fetch + unpack at the current SRCREV + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + + # Upstream adds the file (potentially inside a subdirectory) + full_upstream = os.path.join(self.srcdir, upstream_path) + os.makedirs(os.path.dirname(full_upstream), exist_ok=True) + with open(full_upstream, 'w') as f: + f.write('upstream content\n') + self.git(['add', upstream_path], cwd=self.srcdir) + self.git(['commit', '-m', commit_msg], cwd=self.srcdir) + new_rev = self.git(['rev-parse', 'HEAD'], cwd=self.srcdir).strip() + + # Create the clashing untracked file in the checkout + full_local = os.path.join(unpack_path, local_rel_path) + os.makedirs(os.path.dirname(full_local), exist_ok=True) + with open(full_local, 'w') as f: + f.write('local untracked content\n') + + self.d.setVar('SRCREV', new_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + git_fetcher = ud.method + + # git rebase refuses because the untracked file would be overwritten + with self.assertRaises(bb.fetch2.UnpackError): + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + # Repo must not be left in a mid-rebase state + self.assertFalse(os.path.exists(os.path.join(unpack_path, '.git', 'rebase-merge'))) + self.assertFalse(os.path.exists(os.path.join(unpack_path, '.git', 'rebase-apply'))) + + # The local untracked file must be untouched + self.assertTrue(os.path.exists(full_local)) + with open(full_local) as f: + self.assertEqual(f.read(), 'local untracked content\n') + + # Reset unpackdir for the next sub-case + import shutil as _shutil + _shutil.rmtree(self.unpackdir) + os.makedirs(self.unpackdir) + + # Sub-case 1: top-level file clash + _run_case('newfile.txt', 'newfile.txt', + 'Upstream adds newfile.txt') + + # Sub-case 2: file inside a subdirectory (xxx/somefile) + _run_case('xxx/somefile.txt', 'xxx/somefile.txt', + 'Upstream adds xxx/somefile.txt') + + def test_unpack_update_shallow_clone_fails(self): + """Test that unpack_update raises UnpackError for shallow-tarball checkouts. + + Shallow clones lack full history, which makes an in-place rebase impossible + without network access. After fetching with BB_GIT_SHALLOW=1 the clonedir + is deleted so that unpack() is forced to use the shallow tarball. + A subsequent call to unpack_update() must raise UnpackError and the message + must mention 'shallow clone' so callers can distinguish this case. + """ + self.d.setVar('BB_GIT_SHALLOW', '1') + self.d.setVar('BB_GENERATE_SHALLOW_TARBALLS', '1') + + # First fetch at initial revision + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + # Remove clonedir to force use of shallow tarball + clonedir = os.path.join(self.dldir, 'git2') + if os.path.exists(clonedir): + shutil.rmtree(clonedir) + + fetcher.unpack(self.unpackdir) + + # Update to new revision + self.d.setVar('SRCREV', self.updated_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + # unpack_update should fail for shallow clones + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + git_fetcher = ud.method + + with self.assertRaises(bb.fetch2.UnpackError) as context: + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + self.assertIn("shallow clone", str(context.exception).lower()) + + def test_unpack_update_stale_dldir_remote(self): + """Test that unpack_update raises UnpackError when the dldir remote URL is stale. + + If the clonedir has been removed after the initial unpack (e.g. DL_DIR was + cleaned) the 'dldir' remote URL no longer resolves. The fetch inside + update_mode will fail with a FetchError which must be re-raised as + UnpackError so the caller can fall back to a full re-fetch. + """ + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + + # Advance SRCREV to trigger update_mode + self.d.setVar('SRCREV', self.updated_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + uri = self.d.getVar('SRC_URI') + ud = fetcher.ud[uri] + + # Delete the clonedir and corrupt the dldir remote URL so that + # 'git fetch dldir' fails, simulating a missing or relocated DL_DIR. + shutil.rmtree(ud.clonedir) + self.git(['remote', 'set-url', 'dldir', 'file://' + ud.clonedir], + cwd=unpack_path) + + git_fetcher = ud.method + with self.assertRaises(bb.fetch2.UnpackError): + git_fetcher.unpack_update(ud, self.unpackdir, self.d) + + def test_fetch_unpack_update_toplevel_api(self): + """Test that the top-level Fetch.unpack_update() dispatches to Git.unpack_update(). + + Callers such as bitbake-setup use fetcher.unpack_update(root) rather than + calling the method on the Git fetcher directly. Verify that the public API + works end-to-end: fetch at initial_rev, unpack, advance to updated_rev, + fetch again, then call fetcher.unpack_update(root) and confirm the content + is updated. + """ + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + unpack_path = os.path.join(self.unpackdir, 'git') + with open(os.path.join(unpack_path, 'testfile.txt')) as f: + self.assertEqual(f.read(), 'initial content\n') + + self.d.setVar('SRCREV', self.updated_rev) + fetcher = bb.fetch2.Fetch([self.d.getVar('SRC_URI')], self.d) + fetcher.download() + + # Use the public Fetch.unpack_update() rather than the method directly + fetcher.unpack_update(self.unpackdir) + + with open(os.path.join(unpack_path, 'testfile.txt')) as f: + self.assertEqual(f.read(), 'updated content\n') From patchwork Sun Mar 22 19:34:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83055D58CAB for ; Sun, 22 Mar 2026 19:35:09 +0000 (UTC) Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net [185.136.64.225]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.1876.1774208105448739707 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=cGMuPNWO; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.225, mailfrom: fm-1329275-20260322193502491726ef2900020721-scfcsa@rts-flowmailer.siemens.com) Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 20260322193502491726ef2900020721 for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=lQl+SDuMsEQlaWthQityfwqt4cL3OxMVKG3k/JLkdZ4=; b=cGMuPNWOlO4hvXxsN43DxfdnUvOQzqhQgfLMctSsduDpDGN0BzqUXYNb2uyPZyHnpFJXT5 GTIgdAAiGfF/3eeEUgXfrsZmTS2I1KG13c6pGlpc1qWDE59a2K683YTrY3bH1eth6T4UO6cd 2iXvlG0RaLeQM5cfqQoZrgwMUnKtuzTpjkPnY8jppkcYYJbAJ4Yrw/sXXT3GHcPTmjuWjpuj HQ3/FeCcWXShqira0cdkbnL+QSc7pm4l2StnusDcFU+xFQYjtaNoox3o6+OkJi9oL1GKqlLY O8PKS38z+/hfUQX7AMTn5bLuCzbpYZXlQNf3xO9gcVzHLcdlO+0/7ueQ==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 2/9] bitbake-setup: always restore sys.stdout Date: Sun, 22 Mar 2026 20:34:13 +0100 Message-ID: <20260322193440.870120-3-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19198 From: Adrian Freihofer While working on the bitbake-setup update with a non destructive fetcher, I noticed that if the fetcher raises an exception, sys.stdout is not restored, which can lead to issues in the rest of the code. This change ensures that sys.stdout is always restored, even if an exception occurs during the fetch and unpack process. Showing the full Traceback of the error would be a bit confusing because it happened in code which is not yet ready for review, but the final error was: ValueError: I/O operation on closed file. and this little change seems to be a reasonable improvement to avoid such issues. Signed-off-by: Adrian Freihofer --- bin/bitbake-setup | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/bin/bitbake-setup b/bin/bitbake-setup index dcad9c169..e8d520687 100755 --- a/bin/bitbake-setup +++ b/bin/bitbake-setup @@ -793,9 +793,11 @@ def do_fetch(fetcher, dir): with open(fetchlog, 'a') as f: oldstdout = sys.stdout sys.stdout = f - fetcher.download() - fetcher.unpack_update(dir) - sys.stdout = oldstdout + try: + fetcher.download() + fetcher.unpack_update(dir) + finally: + sys.stdout = oldstdout def update_registry(registry, cachedir, d): registrydir = 'configurations' From patchwork Sun Mar 22 19:34:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 465DBD58CA7 for ; Sun, 22 Mar 2026 19:35:09 +0000 (UTC) Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net [185.136.64.228]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.1877.1774208105448820059 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=MADd5C33; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.228, mailfrom: fm-1329275-202603221935027edf84201c00020717-giry6m@rts-flowmailer.siemens.com) Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 202603221935027edf84201c00020717 for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=ZvsrtByuHhRGhYvFKB/jsVqT3GPrbDYzgxb3raFvR1M=; b=MADd5C33eWAgJMQSETxPADK/1ROGsbqhg1lJIQWy4DbP925ejcZsepBC9xgQ5PLjrJXFR3 kSul56qFCAChK0bzRtxGkTZE3rcujUB9H8TGfJiTALX/zedSh7vcwUrQYTYJOwe9lbcf/4tw 9OcuwMLwZJ0hZVUOdi+BSsxH8azjwinq/mA5FyOB4eedUVE6kzIpd55Yte1w5UFoNj8yiV9T 5bsMMWv77FMGIZ43kGeoP2gi359nbcqXpij6lQuLH7LRfsQHGv5vdOfW1egTbcQmSWQ3Hse9 jLCHRm/19nEgp70ahyYzXWOtUhI5UXSqwOMk0U0ocIbqrXVRYKnDWniw==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 3/9] tests/setup: cleanup imports Date: Sun, 22 Mar 2026 20:34:14 +0100 Message-ID: <20260322193440.870120-4-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19200 From: Adrian Freihofer Add missing top-level imports for os, stat, bb and bb.process and remove the redundant inline 'import os' and 'import stat' that were inside individual methods. Signed-off-by: Adrian Freihofer --- lib/bb/tests/setup.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/lib/bb/tests/setup.py b/lib/bb/tests/setup.py index a66f05b36..d52a81395 100644 --- a/lib/bb/tests/setup.py +++ b/lib/bb/tests/setup.py @@ -5,9 +5,13 @@ # from bb.tests.fetch import FetcherTest -import json -import hashlib +import bb +import bb.process import glob +import hashlib +import json +import os +import stat from bb.tests.support.httpserver import HTTPService class BitbakeSetupTest(FetcherTest): @@ -208,7 +212,6 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"])) with open(fullname, 'w') as f: f.write(content) if script: - import stat st = os.stat(fullname) os.chmod(fullname, st.st_mode | stat.S_IEXEC) self.git('add {}'.format(name), cwd=self.testrepopath) @@ -279,7 +282,6 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"])) def test_setup(self): # unset BBPATH to ensure tests run in isolation from the existing bitbake environment - import os if 'BBPATH' in os.environ: del os.environ['BBPATH'] From patchwork Sun Mar 22 19:34:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CB06D58CA9 for ; Sun, 22 Mar 2026 19:35:09 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.1828.1774208105449174887 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=eDZPZMX/; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-202603221935026b3758b61900020759-mp2qfr@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 202603221935026b3758b61900020759 for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=tHatzJkj8v41S00Pvw0rYn2RiqmtE0gHkGSd5U3RR+E=; b=eDZPZMX/o4lw0BCUi+JN8RYsyJnJ4JrplYnfiYIA9R/8Fr4nRy8yyLHMj3gKb0FcK2qQq+ QSiGUT38NLj6L032E3iOh2ZsOeu3YXbWd7SuigENt+neLuNxvY8ESEJ5d3bTbkS0Ox8+ps0P eWaXbEMNvPIjRjusV3rcZZeyJNf9FqaTZCyWpsXTDQLGxcdcnYc5q6kT+DbednH/tXT2IOv0 hQqXTFoO0O347QcXMS6Klk/syBM+rJv4/CBUDIObvUz1gItblIxw0VKsQ88uVjGVkIHmd91E lkMyddr2zm/i0dbiQzoH+zam+JsdLQB9C60DH6YyKrNnJ2Er1SCXTzHg==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 4/9] tests/setup: fix dead check_setupdir_files guards Date: Sun, 22 Mar 2026 20:34:15 +0100 Message-ID: <20260322193440.870120-5-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19199 From: Adrian Freihofer Two conditions in check_setupdir_files used wrong key names and so never triggered: - 'oe-fragment' -> 'oe-fragments' (plural): fragment-existence assertions were never reached for any variant that has fragments. - 'bb-environment-passthrough' -> 'bb-env-passthrough-additions': BB_ENV_PASSTHROUGH_ADDITIONS assertions were never reached for the gizmo-env-passthrough variant. Also drop the redundant .keys() call on both guards. Signed-off-by: Adrian Freihofer --- lib/bb/tests/setup.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/bb/tests/setup.py b/lib/bb/tests/setup.py index d52a81395..1af3d8b50 100644 --- a/lib/bb/tests/setup.py +++ b/lib/bb/tests/setup.py @@ -260,11 +260,11 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"])) ) self.assertIn(filerelative_layer, bblayers) - if 'oe-fragment' in bitbake_config.keys(): + if 'oe-fragments' in bitbake_config: for f in bitbake_config["oe-fragments"]: self.assertTrue(os.path.exists(os.path.join(bb_conf_path, f))) - if 'bb-environment-passthrough' in bitbake_config.keys(): + if 'bb-env-passthrough-additions' in bitbake_config: with open(os.path.join(bb_build_path, 'init-build-env'), 'r') as f: init_build_env = f.read() self.assertTrue('BB_ENV_PASSTHROUGH_ADDITIONS' in init_build_env) From patchwork Sun Mar 22 19:34:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16AAFC61DE7 for ; Sun, 22 Mar 2026 19:35:08 +0000 (UTC) Received: from mta-65-225.siemens.flowmailer.net (mta-65-225.siemens.flowmailer.net [185.136.65.225]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.1879.1774208105449107866 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=JFLT8R5B; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.225, mailfrom: fm-1329275-20260322193502de1edffa0a000207f7-8mtxqw@rts-flowmailer.siemens.com) Received: by mta-65-225.siemens.flowmailer.net with ESMTPSA id 20260322193502de1edffa0a000207f7 for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=RVLQ4C+UbxiW6niKodR5U+DE8mzTjaaQsYbv/zRg7T0=; b=JFLT8R5B3ZI16z38iAM9QWB4hqf3ZX9LxXnxW6bsIzaX8mZBldqENgBEOf+BSRZ8BIHGXO +PE/XNZgjrjFl9Gjpt4uHdUmrHyccLjXydGBa3laxvk1x9O6ncm9W5j9+LXfFFOMaPwPnG1C EcaAzkfSR7V59DZfIus9WlrX8gfDQim/dAz7Vpn+OHLTYHNgN/8BEkyjCr/4yNfUimhAImbg Efug2qs+FHi2Ra8MONTu1ssmXrZiDD0JguI5gj5LFVuaBEMSdIjqOvDSAnbaNt3V28SF6tuz CQ7rJrwEhAmHhkMWyYAZZQEkyoBp6iCMa5ZCaACYuBkZpyzVJTCyyU3g==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 5/9] bitbake-setup: report local modifications blocking an update Date: Sun, 22 Mar 2026 20:34:16 +0100 Message-ID: <20260322193440.870120-6-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19197 From: Adrian Freihofer When unpack_update() detects uncommitted changes in a layer repository it raises an UnpackError, which previously propagated as an unhandled exception with an opaque traceback. Catch the UnpackError in _checkout_git_remote and re-raise it as a plain Exception with a message that: - names the affected source and its path - tells the user to commit, stash or discard the changes - includes the original error detail for diagnostics Signed-off-by: Adrian Freihofer --- bin/bitbake-setup | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/bin/bitbake-setup b/bin/bitbake-setup index e8d520687..6d24f8fcc 100755 --- a/bin/bitbake-setup +++ b/bin/bitbake-setup @@ -182,7 +182,14 @@ def checkout_layers(layers, confdir, layerdir, d): else: src_uri = f"{fetchuri};protocol={prot};rev={rev};nobranch=1;destsuffix={repodir}" fetcher = bb.fetch.Fetch([src_uri], d) - do_fetch(fetcher, layerdir) + repodir_path = os.path.join(layerdir, repodir) + try: + do_fetch(fetcher, layerdir) + except bb.fetch2.UnpackError as e: + raise Exception( + "Cannot update source '{}' in {} because it has local modifications.\n" + "Please commit, stash or discard your changes and re-run the update.\n" + "Details: {}".format(r_name, repodir_path, e)) from None urldata = fetcher.ud[src_uri] revision = urldata.revision layers_fixed_revisions[r_name]['git-remote']['rev'] = revision From patchwork Sun Mar 22 19:34:17 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41C61D58B02 for ; Sun, 22 Mar 2026 19:35:08 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.1878.1774208105448860720 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=EEX7TjZY; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-20260322193502cb1de83b3e00020764-x8lcne@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 20260322193502cb1de83b3e00020764 for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=y/ZJkYoFmkQPEd+6iBDi9gIAMJjHJx4RfN3Ndwc97Fk=; b=EEX7TjZY2CQpOhv5Uv/qou68Eh4G3MkPQBOQY4kJNKkRjhaPwH5P5XO/nncwqntbrHxSHd 1YZ5sw+yyVGBe4pOZL7S/kKOoYQHZCeWF7rAwd7/2e4OqsDA8uQ6O/wt+HzGT3WbqWhsqiLe 00qXZtyLx8caNiASjBS/hNXOQb8Und3GFVfxQZbuCJNkmZO0vKIXYJw5ec7q1k9EZVkGXffH 326cAuN4vBi3iyo/PJr/IDLZZe6p09ROEwbZwuAIn9oilpKRR4Tgpvy4dbqj6AhhvI51y+TT iiGsfW8mC9p4s0OvznlWOypVosF+n2MKdC3Eu3KdaCSV5Tkegbyq+o4A==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 6/9] bitbake-setup: generate config files for VSCode Date: Sun, 22 Mar 2026 20:34:17 +0100 Message-ID: <20260322193440.870120-7-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19204 From: Adrian Freihofer This change introduces a function to generate a VSCode workspace file (`bitbake.code-workspace`). This workspace file is preferred over a project-specific `.vscode/settings.json` for several reasons: - It allows for a multi-root workspace, which is ideal for a bitbake project structure setup with bitbake-setup. This enables including all layer repositories and the build configuration directory as top-level folders in the explorer. - The workspace file can be located at the top level of the setup, outside of any version-controlled source directory. This avoids cluttering the git repositories with editor-specific configuration. - It provides a centralized place for all VSCode settings related to the project, including those for the bitbake extension, Python language server, and file associations, ensuring a consistent development environment for all users of the project. The Python analysis paths (`python.analysis.extraPaths`) are configured with absolute paths. This is a workaround for a limitation in the Pylance extension, which does not correctly resolve `${workspaceFolder:...}` variables in a multi-root workspace context for import resolution. Using absolute paths ensures that Pylance can find all necessary modules from the various layers. There is room for improvement. Starting a terminal (bitbake or any other) is cumbersome, as VSCode wants to start it for one of the layers rather than the build directory. Signed-off-by: Adrian Freihofer --- bin/bitbake-setup | 200 ++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 185 insertions(+), 15 deletions(-) diff --git a/bin/bitbake-setup b/bin/bitbake-setup index 6d24f8fcc..695043378 100755 --- a/bin/bitbake-setup +++ b/bin/bitbake-setup @@ -246,7 +246,7 @@ bitbake-setup init -L {} /path/to/repo/checkout""".format( return layers_fixed_revisions -def setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_conf): +def setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_conf, init_vscode=False): def _setup_build_conf(layers, filerelative_layers, build_conf_dir): os.makedirs(build_conf_dir) layers_s = [] @@ -376,7 +376,7 @@ def setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_c logger.plain('New bitbake configuration from upstream is the same as the current one, no need to update it.') shutil.rmtree(bitbake_confdir) os.rename(backup_bitbake_confdir, bitbake_confdir) - return + return bitbake_builddir, init_script logger.plain('Upstream bitbake configuration changes were found:') logger.plain(conf_diff) @@ -391,24 +391,30 @@ def setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_c logger.plain(f'Leaving the upstream configuration in {upstream_bitbake_confdir}') os.rename(bitbake_confdir, upstream_bitbake_confdir) os.rename(backup_bitbake_confdir, bitbake_confdir) - return + return bitbake_builddir, init_script logger.plain('Applying upstream bitbake configuration changes') logger.plain(f'Leaving the previous configuration in {backup_bitbake_confdir}') fragment_note = "Run 'bitbake-config-build enable-fragment ' to enable additional fragments or replace built-in ones (e.g. machine/ or distro/ to change MACHINE or DISTRO)." + workspace_file = os.path.join(setupdir, "bitbake.code-workspace") + + readme_extra = "" + if init_vscode: + readme_extra = "\n\nTo edit the code in VSCode, open the workspace: code {}\n".format(workspace_file) + readme = """{}\n\nAdditional information is in {} and {}\n Source the environment using '. {}' to run builds from the command line.\n {}\n -The bitbake configuration files (local.conf, bblayers.conf and more) can be found in {}/conf -""".format( +The bitbake configuration files (local.conf, bblayers.conf and more) can be found in {}/conf{}""".format( bitbake_config["description"], os.path.join(bitbake_builddir,'conf/conf-summary.txt'), os.path.join(bitbake_builddir,'conf/conf-notes.txt'), init_script, fragment_note, - bitbake_builddir + bitbake_builddir, + readme_extra ) readme_file = os.path.join(bitbake_builddir, "README") with open(readme_file, 'w') as f: @@ -419,6 +425,10 @@ The bitbake configuration files (local.conf, bblayers.conf and more) can be foun logger.plain("To run builds, source the environment using\n . {}\n".format(init_script)) logger.plain("{}\n".format(fragment_note)) logger.plain("The bitbake configuration files (local.conf, bblayers.conf and more) can be found in\n {}/conf\n".format(bitbake_builddir)) + if init_vscode: + logger.plain("To edit the code in VSCode, open the workspace:\n code {}\n".format(workspace_file)) + + return bitbake_builddir, init_script def get_registry_config(registry_path, id): for root, dirs, files in os.walk(registry_path): @@ -434,14 +444,15 @@ def merge_overrides_into_sources(sources, overrides): layers[k] = v return layers -def update_build(config, confdir, setupdir, layerdir, d, update_bb_conf="prompt"): +def update_build(config, confdir, setupdir, layerdir, d, update_bb_conf="prompt", init_vscode=False): layer_config = merge_overrides_into_sources(config["data"]["sources"], config["source-overrides"]["sources"]) sources_fixed_revisions = checkout_layers(layer_config, confdir, layerdir, d) bitbake_config = config["bitbake-config"] thisdir = os.path.dirname(config["path"]) if config["type"] == 'local' else None - setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_conf) + bitbake_builddir, init_script = setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_conf, init_vscode) write_sources_fixed_revisions(confdir, layerdir, sources_fixed_revisions) commit_config(confdir) + return bitbake_builddir, init_script def int_input(allowed_values, prompt=''): n = None @@ -628,6 +639,151 @@ def obtain_overrides(args): return overrides +def configure_vscode(setupdir, layerdir, builddir, init_script): + """ + Configure the VSCode environment by creating or updating a workspace file. + + Create or update a bitbake.code-workspace file with folders for the layers and build/conf. + Managed folders are regenerated; user-added folders are kept. Settings are merged, with + managed keys (bitbake.*, python extra paths) always overwritten. + """ + logger.debug("configure_vscode: setupdir={}, layerdir={}, builddir={}, init_script={}".format( + setupdir, layerdir, builddir, init_script)) + + # Get git repository directories + git_repos = [] + if os.path.exists(layerdir): + for entry in os.listdir(layerdir): + entry_path = os.path.join(layerdir, entry) + if os.path.isdir(entry_path) and not os.path.islink(entry_path): + # Check if it's a git repository + if os.path.exists(os.path.join(entry_path, '.git')): + git_repos.append(entry) + logger.debug("configure_vscode: found {} git repos: {}".format(len(git_repos), git_repos)) + + conf_path = os.path.relpath(os.path.join(builddir, "conf"), setupdir) + repo_paths = [os.path.relpath(os.path.join(layerdir, repo), setupdir) for repo in git_repos] + logger.debug("configure_vscode: conf_path={}, repo_paths={}".format(conf_path, repo_paths)) + + # Load existing workspace + workspace_file = os.path.join(setupdir, "bitbake.code-workspace") + workspace = { + "extensions": { + "recommendations": [ + "yocto-project.yocto-bitbake" + ] + } + } + if os.path.exists(workspace_file): + logger.debug("configure_vscode: loading existing workspace file: {}".format(workspace_file)) + try: + with open(workspace_file, 'r') as f: + workspace = json.load(f) + logger.debug("configure_vscode: loaded workspace with {} folders, {} settings".format( + len(workspace.get("folders", [])), len(workspace.get("settings", {})))) + except (json.JSONDecodeError, OSError) as e: + logger.error( + "Unable to read existing workspace file {}: {}. Skipping update.".format( + workspace_file, str(e) + ) + ) + return + else: + logger.debug("configure_vscode: creating new workspace file: {}".format(workspace_file)) + + # Update folders + existing_folders = workspace.get("folders", []) + new_folders = [{"name": "conf", "path": conf_path}] + for rp in repo_paths: + repo_name = os.path.basename(rp) + new_folders.append({"name": repo_name, "path": rp}) + # Keep any user-added folders that are not managed + managed_paths = {f["path"] for f in new_folders} + for f in existing_folders: + if f["path"] not in managed_paths: + new_folders.append(f) + logger.debug("configure_vscode: keeping user-added folder: {}".format(f["path"])) + workspace["folders"] = new_folders + logger.debug("configure_vscode: updated workspace with {} folders".format(len(new_folders))) + + # Build Python extra paths for each layer - only check top level of each repo + extra_paths = [] + subdirs_to_check = ['lib', 'scripts'] + for repo in git_repos: + repo_path_abs = os.path.join(layerdir, repo) + for subdir in subdirs_to_check: + sub_path = os.path.join(repo_path_abs, subdir) + if os.path.isdir(sub_path): + extra_paths.append(sub_path) + + # Update settings + existing_settings = workspace.get("settings", {}) + new_settings = { + "bitbake.disableConfigModification": True, + "bitbake.pathToBitbakeFolder": os.path.join(layerdir, "bitbake"), + "bitbake.pathToBuildFolder": builddir, + "bitbake.pathToEnvScript": init_script, + "bitbake.workingDirectory": builddir, + "files.associations": { + "*.conf": "bitbake", + "*.inc": "bitbake" + }, + "files.exclude": { + "**/.git/**": True + }, + "search.exclude": { + "**/.git/**": True, + "**/logs/**": True + }, + "files.watcherExclude": { + "**/.git/**": True, + "**/logs/**": True + }, + "python.analysis.exclude": [ + "**/.git/**", + "**/logs/**" + ], + "python.autoComplete.extraPaths": extra_paths, + "python.analysis.extraPaths": extra_paths + } + + # Merge settings: add missing, always update bitbake paths and python extra paths + for key, value in new_settings.items(): + if key not in existing_settings: + existing_settings[key] = value + elif key.startswith("bitbake.") or key in [ + "python.autoComplete.extraPaths", + "python.analysis.extraPaths", + ]: + # Always replace - these are managed/machine-generated settings + existing_settings[key] = value + elif key in [ + "files.associations", + "files.exclude", + "search.exclude", + "files.watcherExclude", + "python.analysis.exclude", + ]: + # For dicts and lists, merge new values in without removing user additions + if isinstance(value, dict): + if not isinstance(existing_settings[key], dict): + existing_settings[key] = {} + for k, v in value.items(): + if k not in existing_settings[key]: + existing_settings[key][k] = v + elif isinstance(value, list): + if not isinstance(existing_settings[key], list): + existing_settings[key] = [] + for item in value: + if item not in existing_settings[key]: + existing_settings[key].append(item) + + workspace["settings"] = existing_settings + logger.debug("configure_vscode: merged settings, total {} keys".format(len(existing_settings))) + + with open(workspace_file, 'w') as f: + json.dump(workspace, f, indent=4) + logger.debug("configure_vscode: wrote workspace file: {}".format(workspace_file)) def init_config(top_dir, settings, args): create_siteconf(top_dir, args.non_interactive, settings) @@ -669,7 +825,8 @@ def init_config(top_dir, settings, args): setup_dir_name = n setupdir = os.path.join(os.path.abspath(top_dir), setup_dir_name) - if os.path.exists(os.path.join(setupdir, "layers")): + layerdir = os.path.join(setupdir, "layers") + if os.path.exists(layerdir): logger.info(f"Setup already initialized in:\n {setupdir}\nUse 'bitbake-setup status' to check if it needs to be updated, or 'bitbake-setup update' to perform the update.\nIf you would like to start over and re-initialize in this directory, remove it, and run 'bitbake-setup init' again.") return @@ -683,7 +840,6 @@ def init_config(top_dir, settings, args): os.makedirs(setupdir, exist_ok=True) confdir = os.path.join(setupdir, "config") - layerdir = os.path.join(setupdir, "layers") os.makedirs(confdir) os.makedirs(layerdir) @@ -697,7 +853,10 @@ def init_config(top_dir, settings, args): bb.event.register("bb.build.TaskProgress", handle_task_progress, data=d) write_upstream_config(confdir, upstream_config) - update_build(upstream_config, confdir, setupdir, layerdir, d, update_bb_conf="yes") + bitbake_builddir, init_script = update_build(upstream_config, confdir, setupdir, layerdir, d, update_bb_conf="yes", init_vscode=args.init_vscode) + + if args.init_vscode: + configure_vscode(setupdir, layerdir, bitbake_builddir, init_script) bb.event.remove("bb.build.TaskProgress", None) @@ -773,8 +932,10 @@ def build_status(top_dir, settings, args, d, update=False): if config_diff: logger.plain('\nConfiguration in {} has changed:\n{}'.format(setupdir, config_diff)) if update: - update_build(new_upstream_config, confdir, setupdir, layerdir, d, - update_bb_conf=args.update_bb_conf) + bitbake_builddir, init_script = update_build(new_upstream_config, confdir, setupdir, layerdir, d, + update_bb_conf=args.update_bb_conf, init_vscode=args.init_vscode) + if args.init_vscode: + configure_vscode(setupdir, layerdir, bitbake_builddir, init_script) else: bb.process.run('git -C {} restore config-upstream.json'.format(confdir)) return @@ -782,11 +943,16 @@ def build_status(top_dir, settings, args, d, update=False): layer_config = merge_overrides_into_sources(current_upstream_config["data"]["sources"], current_upstream_config["source-overrides"]["sources"]) if are_layers_changed(layer_config, layerdir, d): if update: - update_build(current_upstream_config, confdir, setupdir, layerdir, - d, update_bb_conf=args.update_bb_conf) + bitbake_builddir, init_script = update_build(current_upstream_config, confdir, setupdir, layerdir, + d, update_bb_conf=args.update_bb_conf, init_vscode=args.init_vscode) + if args.init_vscode: + configure_vscode(setupdir, layerdir, bitbake_builddir, init_script) return logger.plain("\nConfiguration in {} has not changed.".format(setupdir)) + if update and args.init_vscode: + configure_vscode(setupdir, layerdir, os.path.join(setupdir, "build"), + os.path.join(setupdir, "build", "init-build-env")) def build_update(top_dir, settings, args, d): build_status(top_dir, settings, args, d, update=True) @@ -1093,6 +1259,8 @@ def main(): parser_init.add_argument('--skip-selection', action='append', help='Do not select and set an option/fragment from available choices; the resulting bitbake configuration may be incomplete.') parser_init.add_argument('-L', '--use-local-source', default=[], action='append', nargs=2, metavar=('SOURCE_NAME', 'PATH'), help='Symlink local source into a build, instead of getting it as prescribed by a configuration (useful for local development).') + parser_init.add_argument('--init-vscode', action=argparse.BooleanOptionalAction, default=bool(shutil.which('code')), + help='Generate VSCode workspace configuration (default: %(default)s)') parser_init.set_defaults(func=init_config) parser_status = subparsers.add_parser('status', help='Check if the setup needs to be synchronized with configuration') @@ -1102,6 +1270,8 @@ def main(): parser_update = subparsers.add_parser('update', help='Update a setup to be in sync with configuration') add_setup_dir_arg(parser_update) parser_update.add_argument('--update-bb-conf', choices=['prompt', 'yes', 'no'], default='prompt', help='Update bitbake configuration files (bblayers.conf, local.conf) (default: prompt)') + parser_update.add_argument('--init-vscode', action=argparse.BooleanOptionalAction, default=bool(shutil.which('code')), + help='Generate VSCode workspace configuration (default: %(default)s)') parser_update.set_defaults(func=build_update) parser_install_buildtools = subparsers.add_parser('install-buildtools', help='Install buildtools which can help fulfil missing or incorrect dependencies on the host machine') From patchwork Sun Mar 22 19:34:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60991D58CA0 for ; Sun, 22 Mar 2026 19:35:08 +0000 (UTC) Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net [185.136.65.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.1831.1774208105449775212 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=Kxrn9Pv+; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.227, mailfrom: fm-1329275-20260322193502a90c4320fa00020745-jn9jkx@rts-flowmailer.siemens.com) Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20260322193502a90c4320fa00020745 for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=JYqenwduu29YHus0kS7l40H5FOOZA3Wj0p0HJBu2N6A=; b=Kxrn9Pv+UfKX4K4WcDosBelm66QlPnDjo3ZKV/y3WFO/PdvqPX57Co8PsGu22+3Nqe+KNv pG7+WwSvJ2lu2tvAhNFziR+SnTGE0OKloR7LK4Xp/w3TljdKgXDe8KTKQgraN2kppf58Zja9 4/dE0/MJi6ADzddNz0a+qIGUewd/DC8ECO5KfeSuQS3DvWttDCNVjTWdOhk6LWjx3ojyuw/j bMCHJOZxOp8IJggdGTxK+V89Q674/hiOTRGI4XZuVf8qaLFo7YZaHpQO6OMXibK0VybR+IYK 9c6XBxwCwYhFHjOTiaL8ZgDIWkBc8wKr6r4E4J6xmCZc0m8o92GPNqKg==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 7/9] tests/setup: add test_vscode for VSCode workspace generation Date: Sun, 22 Mar 2026 20:34:18 +0100 Message-ID: <20260322193440.870120-8-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19203 From: Adrian Freihofer Add test_vscode to BitbakeSetupTest covering the --init-vscode option introduced in 'bitbake-setup: generate config files for VSCode': - init --init-vscode creates bitbake.code-workspace with the expected top-level structure (folders, settings, extensions). - Folders list conf and each non-symlink git repo in layers/; all paths are relative. - Bitbake extension settings (pathToBuildFolder, pathToEnvScript, disableConfigModification) are set correctly. - file associations (*.conf, *.inc) and python.analysis.extraPaths / python.autoComplete.extraPaths are populated. - init --no-init-vscode does not create a workspace file. - update --init-vscode after a layer change preserves user-added folders and settings while updating managed ones. - update --init-vscode with a corrupt workspace file logs an error and leaves the file unchanged. - update --init-vscode recreates the workspace file when it was manually deleted. Signed-off-by: Adrian Freihofer --- lib/bb/tests/setup.py | 102 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/lib/bb/tests/setup.py b/lib/bb/tests/setup.py index 1af3d8b50..1bcefd698 100644 --- a/lib/bb/tests/setup.py +++ b/lib/bb/tests/setup.py @@ -538,3 +538,105 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"])) custom_setup_dir = 'special-setup-dir-with-cmdline-overrides' out = self.runbbsetup("init --non-interactive -L test-repo {} --setup-dir-name {} test-config-1 gadget".format(self.testrepopath, custom_setup_dir)) _check_local_sources(custom_setup_dir) + + def test_vscode(self): + if 'BBPATH' in os.environ: + del os.environ['BBPATH'] + os.chdir(self.tempdir) + + self.runbbsetup("settings set default registry 'git://{};protocol=file;branch=master;rev=master'".format(self.registrypath)) + self.add_file_to_testrepo('test-file', 'initial\n') + self.add_json_config_to_registry('test-config-1.conf.json', 'master', 'master') + + # --init-vscode should create bitbake.code-workspace + self.runbbsetup("init --non-interactive --init-vscode test-config-1 gadget") + setuppath = self.get_setup_path('test-config-1', 'gadget') + workspace_file = os.path.join(setuppath, 'bitbake.code-workspace') + self.assertTrue(os.path.exists(workspace_file), + "bitbake.code-workspace should be created with --init-vscode") + + with open(workspace_file) as f: + workspace = json.load(f) + + # top-level structure + self.assertIn('folders', workspace) + self.assertIn('settings', workspace) + self.assertIn('extensions', workspace) + self.assertIn('yocto-project.yocto-bitbake', + workspace['extensions']['recommendations']) + + # folders: conf dir + test-repo (symlinks like oe-init-build-env-dir are skipped) + folder_names = {f['name'] for f in workspace['folders']} + self.assertIn('conf', folder_names) + self.assertIn('test-repo', folder_names) + + # folder paths must be relative so the workspace is portable + for f in workspace['folders']: + self.assertFalse(os.path.isabs(f['path']), + "Folder path should be relative, got: {}".format(f['path'])) + + # bitbake extension settings + settings = workspace['settings'] + self.assertTrue(settings.get('bitbake.disableConfigModification')) + self.assertEqual(settings['bitbake.pathToBuildFolder'], + os.path.join(setuppath, 'build')) + self.assertEqual(settings['bitbake.pathToEnvScript'], + os.path.join(setuppath, 'build', 'init-build-env')) + + # file associations + self.assertIn('*.conf', settings.get('files.associations', {})) + self.assertIn('*.inc', settings.get('files.associations', {})) + + # python extra paths: test-repo/scripts/ exists and should be listed + extra_paths = settings.get('python.analysis.extraPaths', []) + self.assertTrue(any('scripts' in p for p in extra_paths), + "python.analysis.extraPaths should include the scripts dir") + self.assertEqual(settings.get('python.analysis.extraPaths'), + settings.get('python.autoComplete.extraPaths')) + + # --no-init-vscode should NOT create a workspace file + self.runbbsetup("init --non-interactive --no-init-vscode test-config-1 gadget-notemplate") + notemplate_path = self.get_setup_path('test-config-1', 'gadget-notemplate') + self.assertFalse( + os.path.exists(os.path.join(notemplate_path, 'bitbake.code-workspace')), + "bitbake.code-workspace should not be created with --no-init-vscode") + + # update with --init-vscode after a layer change should preserve + # user-added folders and settings while still rewriting managed ones + workspace['folders'].append({"name": "user-folder", "path": "user/custom"}) + workspace['settings']['my.user.setting'] = 'preserved' + with open(workspace_file, 'w') as f: + json.dump(workspace, f, indent=4) + + self.add_file_to_testrepo('test-file', 'updated\n') + os.environ['BBPATH'] = os.path.join(setuppath, 'build') + self.runbbsetup("update --update-bb-conf='no' --init-vscode") + del os.environ['BBPATH'] + + with open(workspace_file) as f: + updated = json.load(f) + self.assertIn('user/custom', {f['path'] for f in updated['folders']}, + "User-added folder was removed during update") + self.assertIn('my.user.setting', updated['settings'], + "User-added setting was removed during update") + + # update with a corrupt workspace file should log an error and leave it unchanged + self.add_file_to_testrepo('test-file', 'updated-again\n') + with open(workspace_file, 'w') as f: + f.write('{invalid json') + os.environ['BBPATH'] = os.path.join(setuppath, 'build') + self.runbbsetup("update --update-bb-conf='no' --init-vscode") + del os.environ['BBPATH'] + with open(workspace_file) as f: + content = f.read() + self.assertEqual(content, '{invalid json', + "Corrupt workspace file should not be modified") + + # update with --init-vscode when nothing has changed should still + # regenerate the workspace file (e.g. after manual deletion) + os.remove(workspace_file) + os.environ['BBPATH'] = os.path.join(setuppath, 'build') + self.runbbsetup("update --update-bb-conf='no' --init-vscode") + del os.environ['BBPATH'] + self.assertTrue(os.path.exists(workspace_file), + "bitbake.code-workspace should be recreated by update --init-vscode even when nothing changed") From patchwork Sun Mar 22 19:34:19 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15701FC72DF for ; Sun, 22 Mar 2026 19:35:08 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.1874.1774208105448540912 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=eQEqh9/w; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-2026032219350270238e7fc7000207a8-tbpq5u@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 2026032219350270238e7fc7000207a8 for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=lB9DJ6WFJsZh2LO8Op8nlrCV/A+o1OEulu+Vx4zPlmU=; b=eQEqh9/wziB9jH/9VfasvYpPsymhpyyWbrpRKC5vKUCn4V3o6GMBTl53pVmN/OaN06iFOf VF7C9NLLKjaKNWP5ep20KEETLwAdBS4rByl0fo/YDGKC5LE+YUL+JOQRpGqeB4yLP1EUUClg MGXeegEESQd1LHfeUJNB7ASjBYkgqyxwAB3lbYRL0V8JPRgrF5L79ku3AwPNud8ISuNfBshD MveC//Wl4PK+AgZkefnTEpII7nKFLMY+xWVw5pdOL1Fr6Rj0W0YnDXhpTca5Fko5WN/LIwWS vrxNwLrXs5KUEIKIO21jx+KOQzQq38Xj+SEEiA17NguRTrtNdRt8CdAA==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 8/9] bitbake-setup: add --rebase-strategy to the update command Date: Sun, 22 Mar 2026 20:34:19 +0100 Message-ID: <20260322193440.870120-9-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19206 From: Adrian Freihofer When unpack_update() raises an UnpackError due to local modifications in a layer repository, the caller now has a choice of how to handle it, controlled by the new --rebase-strategy option on the 'update' subcommand: - stop (default): abort with a clear error message that names the affected source and directory, and suggests --rebase-strategy=backup as a remedy. - backup: rename the directory to a timestamped -backup path to preserve local work, then re-clone from upstream via fetcher.unpack(). The strategy is threaded from the CLI argument through build_status() and update_build() down to checkout_layers(). The choices= list in argparse makes it straightforward to add further strategies (e.g. interactive or AI) in the future. Signed-off-by: Adrian Freihofer --- bin/bitbake-setup | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-) diff --git a/bin/bitbake-setup b/bin/bitbake-setup index 695043378..dc433fa22 100755 --- a/bin/bitbake-setup +++ b/bin/bitbake-setup @@ -166,7 +166,7 @@ def _get_remotes(r_remote): return remotes -def checkout_layers(layers, confdir, layerdir, d): +def checkout_layers(layers, confdir, layerdir, d, rebase_strategy='stop'): def _checkout_git_remote(r_remote, repodir, layers_fixed_revisions): rev = r_remote['rev'] branch = r_remote.get('branch', None) @@ -186,10 +186,20 @@ def checkout_layers(layers, confdir, layerdir, d): try: do_fetch(fetcher, layerdir) except bb.fetch2.UnpackError as e: - raise Exception( - "Cannot update source '{}' in {} because it has local modifications.\n" - "Please commit, stash or discard your changes and re-run the update.\n" - "Details: {}".format(r_name, repodir_path, e)) from None + if rebase_strategy == 'backup': + backup_path = add_unique_timestamp_to_path(repodir_path + '-backup') + logger.warning( + "Source '{}' at {} has local modifications that prevent an in-place update.\n" + "Renaming it to {} to preserve your work, then re-cloning from upstream." + .format(r_name, repodir_path, backup_path)) + os.rename(repodir_path, backup_path) + fetcher.unpack(layerdir) + else: + raise Exception( + "Cannot update source '{}' in {} because it has local modifications.\n" + "Please commit, stash or discard your changes and re-run the update,\n" + "or use --rebase-strategy=backup to back up the directory and re-clone automatically.\n" + "Details: {}".format(r_name, repodir_path, e)) from None urldata = fetcher.ud[src_uri] revision = urldata.revision layers_fixed_revisions[r_name]['git-remote']['rev'] = revision @@ -444,9 +454,9 @@ def merge_overrides_into_sources(sources, overrides): layers[k] = v return layers -def update_build(config, confdir, setupdir, layerdir, d, update_bb_conf="prompt", init_vscode=False): +def update_build(config, confdir, setupdir, layerdir, d, update_bb_conf="prompt", init_vscode=False, rebase_strategy='stop'): layer_config = merge_overrides_into_sources(config["data"]["sources"], config["source-overrides"]["sources"]) - sources_fixed_revisions = checkout_layers(layer_config, confdir, layerdir, d) + sources_fixed_revisions = checkout_layers(layer_config, confdir, layerdir, d, rebase_strategy=rebase_strategy) bitbake_config = config["bitbake-config"] thisdir = os.path.dirname(config["path"]) if config["type"] == 'local' else None bitbake_builddir, init_script = setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_conf, init_vscode) @@ -933,7 +943,7 @@ def build_status(top_dir, settings, args, d, update=False): logger.plain('\nConfiguration in {} has changed:\n{}'.format(setupdir, config_diff)) if update: bitbake_builddir, init_script = update_build(new_upstream_config, confdir, setupdir, layerdir, d, - update_bb_conf=args.update_bb_conf, init_vscode=args.init_vscode) + update_bb_conf=args.update_bb_conf, init_vscode=args.init_vscode, rebase_strategy=args.rebase_strategy) if args.init_vscode: configure_vscode(setupdir, layerdir, bitbake_builddir, init_script) else: @@ -944,7 +954,7 @@ def build_status(top_dir, settings, args, d, update=False): if are_layers_changed(layer_config, layerdir, d): if update: bitbake_builddir, init_script = update_build(current_upstream_config, confdir, setupdir, layerdir, - d, update_bb_conf=args.update_bb_conf, init_vscode=args.init_vscode) + d, update_bb_conf=args.update_bb_conf, init_vscode=args.init_vscode, rebase_strategy=args.rebase_strategy) if args.init_vscode: configure_vscode(setupdir, layerdir, bitbake_builddir, init_script) return @@ -1272,6 +1282,10 @@ def main(): parser_update.add_argument('--update-bb-conf', choices=['prompt', 'yes', 'no'], default='prompt', help='Update bitbake configuration files (bblayers.conf, local.conf) (default: prompt)') parser_update.add_argument('--init-vscode', action=argparse.BooleanOptionalAction, default=bool(shutil.which('code')), help='Generate VSCode workspace configuration (default: %(default)s)') + parser_update.add_argument('--rebase-strategy', choices=['stop', 'backup'], default='stop', + help="What to do when a layer repository has local modifications that prevent " + "an in-place update: 'stop' (default) aborts with an error message; " + "'backup' renames the directory to a timestamped backup and re-clones from upstream.") parser_update.set_defaults(func=build_update) parser_install_buildtools = subparsers.add_parser('install-buildtools', help='Install buildtools which can help fulfil missing or incorrect dependencies on the host machine') From patchwork Sun Mar 22 19:34:20 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 84088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B143D58CA3 for ; Sun, 22 Mar 2026 19:35:08 +0000 (UTC) Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net [185.136.65.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.1830.1774208105449436469 for ; Sun, 22 Mar 2026 12:35:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=ctN8yv/l; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.227, mailfrom: fm-1329275-20260322193502b02f621e720002070f-wg7ysu@rts-flowmailer.siemens.com) Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20260322193502b02f621e720002070f for ; Sun, 22 Mar 2026 20:35:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=ciAU+qIK3242iL4jjD6jZvg+8bknGkNoiz2QpRlcNF0=; b=ctN8yv/lmFQUkQ/SnEGJIKIeYwJLRR9KszOJUZGCOBhq3LBro2orBsZdDtsO6eXk+TbACG T+iC0AOSB1z1+AVvW03c74mvGpBY/tXryg2h7LMMlpqRnaFRpLd9V6i2ltDOmO/q4PgCoJ1I zW6UnOsHbDC22oPSED0MtHQEHdVRqxtvJPXC3AmUTlq1xtWv/XyP+gV9ockUM9H+rpHxdn25 aOdhUebegQSUICjduGUab/Ytiza073o7CXpGNvtp4qqBXcCTa5knobkGmYy7mkt8tg5NT2bn mJ0s/w/Ul9h2xBlMJfPG3KnqcJGLoG2MiGIInbQ23n/bO1cga2Lmrfjg==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 9/9] tests/setup: add test_update_rebase_strategy Date: Sun, 22 Mar 2026 20:34:20 +0100 Message-ID: <20260322193440.870120-10-adrian.freihofer@siemens.com> In-Reply-To: <20260322193440.870120-1-adrian.freihofer@siemens.com> References: <20260322193440.870120-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 22 Mar 2026 19:35:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19202 From: Adrian Freihofer Add a dedicated test for the --rebase-strategy option introduced in 'bitbake-setup: add --rebase-strategy to the update command'. Three scenarios are covered that are not exercised by the existing test_setup: 1. Uncommitted tracked-file change, default 'stop' strategy: update must fail with an error message that names the source and hints at --rebase-strategy=backup; no backup directory is created. 2. Same uncommitted change, 'backup' strategy: the layer directory is renamed to a timestamped backup, the layer is re-cloned from upstream, and the result is clean (upstream content, no local modifications). 3. Committed local change that conflicts with the next upstream commit, 'backup' strategy: instead of the hard rebase-conflict failure that occurs with the default strategy, the conflicted directory is backed up and re-cloned successfully. A small _count_layer_backups() helper is added to the class and reused across scenarios. Signed-off-by: Adrian Freihofer --- lib/bb/tests/setup.py | 88 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/lib/bb/tests/setup.py b/lib/bb/tests/setup.py index 1bcefd698..6339fd5ac 100644 --- a/lib/bb/tests/setup.py +++ b/lib/bb/tests/setup.py @@ -640,3 +640,91 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"])) del os.environ['BBPATH'] self.assertTrue(os.path.exists(workspace_file), "bitbake.code-workspace should be recreated by update --init-vscode even when nothing changed") + + def _count_layer_backups(self, layers_path): + return len([f for f in os.listdir(layers_path) if 'backup' in f]) + + def test_update_rebase_strategy(self): + """Test the --rebase-strategy option for the update command. + + Covers three scenarios not exercised by test_setup: + 1. Uncommitted tracked-file change + default 'stop' strategy → clean error + message that names the source and mentions --rebase-strategy=backup. + 2. Uncommitted tracked-file change + 'backup' strategy → directory is + renamed to a timestamped backup and the layer is re-cloned cleanly. + 3. Committed local change that would cause a rebase conflict + 'backup' + strategy → backup + re-clone instead of a hard failure. + """ + if 'BBPATH' in os.environ: + del os.environ['BBPATH'] + os.chdir(self.tempdir) + + self.runbbsetup("settings set default registry 'git://{};protocol=file;branch=master;rev=master'".format(self.registrypath)) + self.add_file_to_testrepo('test-file', 'initial\n') + self.add_json_config_to_registry('test-config-1.conf.json', 'master', 'master') + self.runbbsetup("init --non-interactive test-config-1 gadget") + + setuppath = self.get_setup_path('test-config-1', 'gadget') + layer_path = os.path.join(setuppath, 'layers', 'test-repo') + layers_path = os.path.join(setuppath, 'layers') + + # Scenario 1: uncommitted tracked change, default 'stop' strategy + # Advance upstream so an update is required. + self.add_file_to_testrepo('test-file', 'upstream-v2\n') + # Modify the same tracked file in the layer without committing. + with open(os.path.join(layer_path, 'test-file'), 'w') as f: + f.write('locally-modified\n') + + os.environ['BBPATH'] = os.path.join(setuppath, 'build') + with self.assertRaises(bb.process.ExecutionError) as ctx: + self.runbbsetup("update --update-bb-conf='no'") + self.assertIn('has local modifications', str(ctx.exception)) + self.assertIn('--rebase-strategy=backup', str(ctx.exception)) + # No backup directory must have been created. + self.assertEqual(self._count_layer_backups(layers_path), 0, + "stop strategy must not create any backup") + + # Scenario 2: same uncommitted change, 'backup' strategy + out = self.runbbsetup("update --update-bb-conf='no' --rebase-strategy=backup") + # One backup directory must now exist. + self.assertEqual(self._count_layer_backups(layers_path), 1, + "backup strategy must create exactly one backup") + # The re-cloned layer must be clean and at the upstream revision. + with open(os.path.join(layer_path, 'test-file')) as f: + self.assertEqual(f.read(), 'upstream-v2\n', + "re-cloned layer must contain the upstream content") + status = self.git('status --porcelain', cwd=layer_path).strip() + self.assertEqual(status, '', + "re-cloned layer must have no local modifications") + del os.environ['BBPATH'] + + # Scenario 3: committed conflicting change, 'backup' strategy + # Re-initialise a fresh setup so we start from a clean state. + self.runbbsetup("init --non-interactive --setup-dir-name rebase-conflict-setup test-config-1 gadget") + conflict_setup = os.path.join(self.tempdir, 'bitbake-builds', 'rebase-conflict-setup') + conflict_layer = os.path.join(conflict_setup, 'layers', 'test-repo') + conflict_layers = os.path.join(conflict_setup, 'layers') + + # Commit a local change that touches the same file as the next upstream commit. + with open(os.path.join(conflict_layer, 'test-file'), 'w') as f: + f.write('conflicting-local\n') + self.git('add test-file', cwd=conflict_layer) + self.git('commit -m "Local conflicting change"', cwd=conflict_layer) + + # Advance upstream with a conflicting edit. + self.add_file_to_testrepo('test-file', 'conflicting-upstream\n') + + os.environ['BBPATH'] = os.path.join(conflict_setup, 'build') + # Default stop strategy must still fail with a conflict error. + with self.assertRaisesRegex(bb.process.ExecutionError, "Merge conflict in test-file"): + self.runbbsetup("update --update-bb-conf='no'") + self.assertEqual(self._count_layer_backups(conflict_layers), 0) + + # Backup strategy must succeed: backup the conflicted dir and re-clone. + self.runbbsetup("update --update-bb-conf='no' --rebase-strategy=backup") + self.assertEqual(self._count_layer_backups(conflict_layers), 1, + "backup strategy must create exactly one backup after a conflict") + with open(os.path.join(conflict_layer, 'test-file')) as f: + self.assertEqual(f.read(), 'conflicting-upstream\n', + "re-cloned layer must contain the upstream content after conflict backup") + del os.environ['BBPATH']