diff mbox series

[1/3] bblayers/setupwriters/oe-local-copy: add a 'local copy' plugin for bitbake-layers create-layer-setup

Message ID 20240611141522.3075000-1-alex.kanavin@gmail.com
State New
Headers show
Series [1/3] bblayers/setupwriters/oe-local-copy: add a 'local copy' plugin for bitbake-layers create-layer-setup | expand

Commit Message

Alexander Kanavin June 11, 2024, 2:15 p.m. UTC
From: Alexander Kanavin <alex@linutronix.de>

This plugin copies all currently configured layer respositories into
a dedicated location on local disk. This is useful for entirely offline
layer replication, when the layers are packed and then unpacked from
an archive, rather than fetched from git (there can be situations
where fetching from git is undesirable or impossible).

This plugin will also be used when replicating an entire yocto build
(with build config and sstate cache), and is an element of that.

This plugin is similar to what esdk does, and it provides that functionality
outside of populate_sdk_ext task. It does not reuse esdk code,
as that simply copies the layer tree; it's better to use 'git clone'
with a path to original repo on local disk, as that will preserve
commit history in the copy.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 .../bblayers/setupwriters/oe-local-copy.py    | 41 +++++++++++++++++++
 meta/lib/oeqa/selftest/cases/bblayers.py      | 13 ++++++
 2 files changed, 54 insertions(+)
 create mode 100644 meta/lib/bblayers/setupwriters/oe-local-copy.py

Comments

patchtest@automation.yoctoproject.org June 11, 2024, 2:38 p.m. UTC | #1
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/1-3-bblayers-setupwriters-oe-local-copy-add-a-local-copy-plugin-for-bitbake-layers-create-layer-setup.patch

FAIL: test shortlog length: Edit shortlog so that it is 90 characters or less (currently 100 characters) (test_mbox.TestMbox.test_shortlog_length)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)

SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!
diff mbox series

Patch

diff --git a/meta/lib/bblayers/setupwriters/oe-local-copy.py b/meta/lib/bblayers/setupwriters/oe-local-copy.py
new file mode 100644
index 00000000000..8c1ccb67368
--- /dev/null
+++ b/meta/lib/bblayers/setupwriters/oe-local-copy.py
@@ -0,0 +1,41 @@ 
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+import logging
+import json
+
+logger = logging.getLogger('bitbake-layers')
+
+def plugin_init(plugins):
+    return OeLocalCopyWriter()
+
+class OeLocalCopyWriter():
+
+    def __str__(self):
+        return "oe-local-copy"
+
+    def do_write(self, parent, args):
+        """ Writes out a local copy of all the metadata layers (and bitbake) included in a current build. """
+        if not os.path.exists(args.destdir):
+            os.makedirs(args.destdir)
+        repos = parent.make_repo_config(args.destdir)
+        if not repos:
+            raise Exception("Could not determine layer sources")
+        output = os.path.join(os.path.abspath(args.destdir), args.output_prefix or "layers")
+        json_f = os.path.join(os.path.abspath(args.destdir), "bundle-repos.json")
+
+        for r in repos.values():
+            r['git-remote']['remotes'] = {"origin":{"uri":r["originpath"]}}
+
+        json_data = {"version":"1.0","sources":repos}
+        with open(json_f, 'w') as f:
+            json.dump(json_data, f, sort_keys=True, indent=4)
+
+        logger.info("Cloning repositories into {}".format(output))
+        bb.process.run('oe-setup-layers --force-bootstraplayer-checkout --destdir {} --jsondata {}'.format(output, json_f))
+
+    def register_arguments(self, parser):
+        pass
diff --git a/meta/lib/oeqa/selftest/cases/bblayers.py b/meta/lib/oeqa/selftest/cases/bblayers.py
index 695d17377d4..8b2bc319d50 100644
--- a/meta/lib/oeqa/selftest/cases/bblayers.py
+++ b/meta/lib/oeqa/selftest/cases/bblayers.py
@@ -240,3 +240,16 @@  class BitbakeLayers(OESelftestTestCase):
         self.assertEqual(first_desc_2, '', "Describe not cleared: '{}'".format(first_desc_2))
         self.assertEqual(second_rev_2, second_rev_1, "Revision should not be updated: '{}'".format(second_rev_2))
         self.assertEqual(second_desc_2, second_desc_1, "Describe should not be updated: '{}'".format(second_desc_2))
+
+    def test_bitbakelayers_setup_localcopy(self):
+        testcopydir = os.path.join(self.builddir, 'test-layer-copy')
+        result = runCmd('bitbake-layers create-layers-setup --writer oe-local-copy {}'.format(testcopydir))
+        oe_core_found = False
+        meta_selftest_found = False
+        for topdir, subdirs, files in os.walk(testcopydir):
+            if topdir.endswith('meta/conf') and 'layer.conf' in files:
+                oe_core_found = True
+            if topdir.endswith('meta-selftest/conf') and 'layer.conf' in files:
+                meta_selftest_found = True
+        self.assertTrue(oe_core_found, "meta/conf/layer.conf not found in {}".format(testcopydir))
+        self.assertTrue(meta_selftest_found, "meta-selftest/conf/layer.conf not found in {}".format(testcopydir))