diff mbox series

[17/24] devtool: deploy: split ssh deployment into a separate function

Message ID 20260830142922.17241-18-adrian.freihofer@siemens.com
State New
Headers show
Series devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes | expand

Commit Message

AdrianF Aug. 30, 2026, 2:28 p.m. UTC
From: Adrian Freihofer <adrian.freihofer@siemens.com>

This is a refactoring preparation for adding a new local deployment
method that will be used for NFS rootfs deployments.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 scripts/lib/devtool/deploy.py | 168 ++++++++++++++++++----------------
 1 file changed, 87 insertions(+), 81 deletions(-)

Comments

patchtest@automation.yoctoproject.org Aug. 30, 2026, 2:56 p.m. UTC | #1
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/17-24-devtool-deploy-split-ssh-deployment-into-a-separate-function.patch

FAIL: test max line length: Patch line too long (current length 247, maximum is 200) (test_metadata.TestMetadata.test_max_line_length)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test auh changelog truncation notice (test_mbox.TestMbox.test_auh_changelog_truncation_notice)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!
diff mbox series

Patch

diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index cf481d6b63..a4fd83d305 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -254,6 +254,91 @@  def deploy(args, config, basepath, workspace):
 
     return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files)
 
+def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files,
+                fakerootcmd, fakerootenv, path, recipe_outdir):
+    """Copy files to target_dir over ssh/scp (user@hostname[:destdir])."""
+    extraoptions = ''
+    if args.no_host_check:
+        extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
+    if not args.show_status:
+        extraoptions += ' -q'
+
+    scp_sshexec = ''
+    ssh_sshexec = 'ssh'
+    if args.ssh_exec:
+        scp_sshexec = "-S %s" % args.ssh_exec
+        ssh_sshexec = args.ssh_exec
+    scp_port = ''
+    ssh_port = ''
+    if args.port:
+        scp_port = "-P %s" % args.port
+        ssh_port = "-p %s" % args.port
+
+    if args.key:
+        extraoptions += ' -i %s' % args.key
+
+    # In order to delete previously deployed files and have the manifest file on
+    # the target, we write out a shell script and then copy it to the target
+    # so we can then run it (piping tar output to it).
+    # (We cannot use scp here, because it doesn't preserve symlinks.)
+    tmpdir = tempfile.mkdtemp(prefix='devtool')
+    try:
+        tmpscript = '/tmp/devtool_deploy.sh'
+        tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list')
+        shellscript = _prepare_remote_script(deploy=True,
+                                            destdir=destdir,
+                                            verbose=args.show_status,
+                                            nopreserve=args.no_preserve,
+                                            nocheckspace=args.no_check_space)
+        # Write out the script to a file
+        with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f:
+            f.write(shellscript)
+        # Write out the file list
+        with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f:
+            f.write('%d\n' % ftotalsize)
+            for fpath, fsize in filelist:
+                f.write('%s %d\n' % (fpath, fsize))
+        # Copy them to the target
+        ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True)
+        if ret != 0:
+            raise DevtoolError('Failed to copy script to %s - rerun with -s to '
+                            'get a complete error message' % args.target)
+    finally:
+        shutil.rmtree(tmpdir)
+
+    # Now run the script. When a package/glob filter narrowed down filelist,
+    # tar is given an explicit list of relative paths (-T) instead of packing
+    # the whole recipe_outdir tree.
+    tar_filelist_path = None
+    try:
+        if allowed_files is not None:
+            tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-')
+            with os.fdopen(tar_fd, 'w') as f:
+                for relpath in tar_relpaths:
+                    # './' prefix matches what 'tar cf - .' itself would produce, which
+                    # the remote script's manifest handling (sed "s!^./!$2!") relies on.
+                    f.write('./' + relpath + '\n')
+            tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path)
+        else:
+            tar_cmd = 'tar cf - .'
+        ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, '%s | %s  %s %s %s \'sh %s %s %s %s\'' % (tar_cmd, ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True)
+    finally:
+        if tar_filelist_path:
+            os.remove(tar_filelist_path)
+    if ret != 0:
+        raise DevtoolError('Deploy failed - rerun with -s to get a complete '
+                        'error message')
+
+    logger.info('Successfully deployed %s' % recipe_outdir)
+
+    files_list = []
+    for root, _, files in os.walk(recipe_outdir):
+        for filename in files:
+            filename = os.path.relpath(os.path.join(root, filename), recipe_outdir)
+            files_list.append(os.path.join(destdir, filename))
+
+    return 0
+
 def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None):
     import math
 
@@ -351,87 +436,8 @@  def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce
             print('  %s' % item)
         return 0
 
-    extraoptions = ''
-    if args.no_host_check:
-        extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
-    if not args.show_status:
-        extraoptions += ' -q'
-
-    scp_sshexec = ''
-    ssh_sshexec = 'ssh'
-    if args.ssh_exec:
-        scp_sshexec = "-S %s" % args.ssh_exec
-        ssh_sshexec = args.ssh_exec
-    scp_port = ''
-    ssh_port = ''
-    if args.port:
-        scp_port = "-P %s" % args.port
-        ssh_port = "-p %s" % args.port
-
-    if args.key:
-        extraoptions += ' -i %s' % args.key
-
-    # In order to delete previously deployed files and have the manifest file on
-    # the target, we write out a shell script and then copy it to the target
-    # so we can then run it (piping tar output to it).
-    # (We cannot use scp here, because it doesn't preserve symlinks.)
-    tmpdir = tempfile.mkdtemp(prefix='devtool')
-    try:
-        tmpscript = '/tmp/devtool_deploy.sh'
-        tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list')
-        shellscript = _prepare_remote_script(deploy=True,
-                                            destdir=destdir,
-                                            verbose=args.show_status,
-                                            nopreserve=args.no_preserve,
-                                            nocheckspace=args.no_check_space)
-        # Write out the script to a file
-        with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f:
-            f.write(shellscript)
-        # Write out the file list
-        with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f:
-            f.write('%d\n' % ftotalsize)
-            for fpath, fsize in filelist:
-                f.write('%s %d\n' % (fpath, fsize))
-        # Copy them to the target
-        ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True)
-        if ret != 0:
-            raise DevtoolError('Failed to copy script to %s - rerun with -s to '
-                            'get a complete error message' % args.target)
-    finally:
-        shutil.rmtree(tmpdir)
-
-    # Now run the script. When a package/glob filter narrowed down filelist,
-    # tar is given an explicit list of relative paths (-T) instead of packing
-    # the whole recipe_outdir tree.
-    tar_filelist_path = None
-    try:
-        if allowed_files is not None:
-            tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-')
-            with os.fdopen(tar_fd, 'w') as f:
-                for relpath in tar_relpaths:
-                    # './' prefix matches what 'tar cf - .' itself would produce, which
-                    # the remote script's manifest handling (sed "s!^./!$2!") relies on.
-                    f.write('./' + relpath + '\n')
-            tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path)
-        else:
-            tar_cmd = 'tar cf - .'
-        ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, '%s | %s  %s %s %s \'sh %s %s %s %s\'' % (tar_cmd, ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True)
-    finally:
-        if tar_filelist_path:
-            os.remove(tar_filelist_path)
-    if ret != 0:
-        raise DevtoolError('Deploy failed - rerun with -s to get a complete '
-                        'error message')
-
-    logger.info('Successfully deployed %s' % recipe_outdir)
-
-    files_list = []
-    for root, _, files in os.walk(recipe_outdir):
-        for filename in files:
-            filename = os.path.relpath(os.path.join(root, filename), recipe_outdir)
-            files_list.append(os.path.join(destdir, filename))
-
-    return 0
+    return _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths,
+                        allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir)
 
 def undeploy(args, config, basepath, workspace):
     """Entry point for the devtool 'undeploy' subcommand"""