From patchwork Thu Mar 31 18:31:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17A9BC433F5 for ; Thu, 31 Mar 2022 18:31:22 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.749.1648751481282267781 for ; Thu, 31 Mar 2022 11:31:21 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9AE58139F for ; Thu, 31 Mar 2022 11:31:20 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4BA4A3F718 for ; Thu, 31 Mar 2022 11:31:20 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 1/7] runfvp: propagate the exit code correctly Date: Thu, 31 Mar 2022 19:31:11 +0100 Message-Id: <20220331183117.22389-1-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:31:22 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/3217 runfvp could encounter an error but the exit code remained as 0. Signed-off-by: Ross Burton --- scripts/runfvp | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/scripts/runfvp b/scripts/runfvp index c20455fd..5cab0949 100755 --- a/scripts/runfvp +++ b/scripts/runfvp @@ -221,6 +221,9 @@ async def start_fvp(cli, console_cb): if await fvp_process.wait() != 0: logger.info(f"{cli[0]} quit with code {fvp_process.returncode}") + return fvp_process.returncode + else: + return 0 def runfvp(cli_args): args, fvp_args = parse_args(cli_args) @@ -234,7 +237,7 @@ def runfvp(cli_args): expected_terminal = config["console"] if not expected_terminal: logger.error("--console used but FVP_CONSOLE not set in machine configuration") - sys.exit(1) + return 1 else: expected_terminal = None @@ -252,9 +255,10 @@ def runfvp(cli_args): # When we can assume Py3.7+, this can simply be asyncio.run() loop = asyncio.get_event_loop() console_cb = expected_terminal and console_started or None - loop.run_until_complete(asyncio.gather(start_fvp(cli, console_cb=console_cb))) + return loop.run_until_complete(start_fvp(cli, console_cb=console_cb)) except asyncio.CancelledError: - pass + # This means telnet exited, which isn't an error + return 0 if __name__ == "__main__": try: @@ -264,6 +268,6 @@ if __name__ == "__main__": if sys.stdin.isatty(): signal.signal(signal.SIGTTOU, signal.SIG_IGN) os.tcsetpgrp(sys.stdin.fileno(), os.getpgrp()) - runfvp(sys.argv[1:]) + sys.exit(runfvp(sys.argv[1:])) except KeyboardInterrupt: pass From patchwork Thu Mar 31 18:31:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ACD9C433EF for ; Thu, 31 Mar 2022 18:31:22 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.704.1648751481561512589 for ; Thu, 31 Mar 2022 11:31:21 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2C1DF13D5 for ; Thu, 31 Mar 2022 11:31:21 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D22D93F718 for ; Thu, 31 Mar 2022 11:31:20 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 2/7] runfvp: check for telnet Date: Thu, 31 Mar 2022 19:31:12 +0100 Message-Id: <20220331183117.22389-2-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331183117.22389-1-ross.burton@arm.com> References: <20220331183117.22389-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:31:22 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/3218 Check for telnet on startup to avoid mysterious failures later when telnet isn't available. Signed-off-by: Ross Burton --- scripts/runfvp | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/scripts/runfvp b/scripts/runfvp index 5cab0949..0ebf873e 100755 --- a/scripts/runfvp +++ b/scripts/runfvp @@ -5,6 +5,7 @@ import collections import json import os import re +import shutil import signal import sys import subprocess @@ -42,7 +43,7 @@ class Terminals: return config.get("RunFVP", "Terminal", fallback=None) def preferred_terminal(self) -> str: - import shlex, shutil + import shlex preferred = self.configured_terminal() if preferred: @@ -233,6 +234,11 @@ def runfvp(cli_args): cli.extend(fvp_args) logger.debug(f"Constructed FVP call: {cli}") + # Check that telnet is present + if not bool(shutil.which("telnet")): + logger.error("Cannot find telnet, this is needed to connect to the FVP.") + return 1 + if args.console: expected_terminal = config["console"] if not expected_terminal: From patchwork Thu Mar 31 18:31:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBD2BC433FE for ; Thu, 31 Mar 2022 18:31:22 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.750.1648751482077540218 for ; Thu, 31 Mar 2022 11:31:22 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B2465139F for ; Thu, 31 Mar 2022 11:31:21 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6464C3F718 for ; Thu, 31 Mar 2022 11:31:21 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 3/7] runfvp: strip all suffixes from the image when calculating .fvpconf name Date: Thu, 31 Mar 2022 19:31:13 +0100 Message-Id: <20220331183117.22389-3-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331183117.22389-1-ross.burton@arm.com> References: <20220331183117.22389-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:31:22 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/3219 Until testimage tells the controller the basename of the image, make sure to strip all suffixes from the image name to get the base name, not just one. Machines such as corstone500 have images called .wic.nopt, so just stripping one isn't sufficient. Signed-off-by: Ross Burton --- meta-arm/lib/oeqa/controllers/fvp.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/meta-arm/lib/oeqa/controllers/fvp.py b/meta-arm/lib/oeqa/controllers/fvp.py index 7136a003..2913f782 100644 --- a/meta-arm/lib/oeqa/controllers/fvp.py +++ b/meta-arm/lib/oeqa/controllers/fvp.py @@ -16,7 +16,9 @@ class OEFVPTarget(oeqa.core.target.ssh.OESSHTarget): **kwargs): super().__init__(logger, target_ip, server_ip, timeout, user, port) image_dir = pathlib.Path(dir_image) - basename = pathlib.Path(rootfs).stem + # rootfs may have multiple extensions so we need to strip *all* suffixes + basename = pathlib.Path(rootfs) + basename = basename.name.replace("".join(basename.suffixes), "") self.fvpconf = image_dir / (basename + ".fvpconf") if not self.fvpconf.exists(): From patchwork Thu Mar 31 18:31:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02618C433F5 for ; Thu, 31 Mar 2022 18:31:28 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web08.738.1648751482552970124 for ; Thu, 31 Mar 2022 11:31:22 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4485B13D5 for ; Thu, 31 Mar 2022 11:31:22 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EAD733F718 for ; Thu, 31 Mar 2022 11:31:21 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 4/7] arm/oeqa: add basic runfvp test cases Date: Thu, 31 Mar 2022 19:31:14 +0100 Message-Id: <20220331183117.22389-4-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331183117.22389-1-ross.burton@arm.com> References: <20220331183117.22389-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:31:28 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/3220 Add rudimentary but functional test cases for runfvp. Signed-off-by: Ross Burton --- meta-arm/lib/oeqa/selftest/cases/runfvp.py | 51 +++++++++++++++++++ .../oeqa/selftest/cases/tests/auto-basic.json | 3 ++ .../oeqa/selftest/cases/tests/auto-basic.sh | 11 ++++ .../selftest/cases/tests/auto-parameters.json | 7 +++ .../lib/oeqa/selftest/cases/tests/mock-fvp.py | 22 ++++++++ .../selftest/cases/tests/test-parameter.json | 6 +++ .../selftest/cases/tests/test-parameters.py | 1 + 7 files changed, 101 insertions(+) create mode 100644 meta-arm/lib/oeqa/selftest/cases/runfvp.py create mode 100644 meta-arm/lib/oeqa/selftest/cases/tests/auto-basic.json create mode 100755 meta-arm/lib/oeqa/selftest/cases/tests/auto-basic.sh create mode 100644 meta-arm/lib/oeqa/selftest/cases/tests/auto-parameters.json create mode 100755 meta-arm/lib/oeqa/selftest/cases/tests/mock-fvp.py create mode 100644 meta-arm/lib/oeqa/selftest/cases/tests/test-parameter.json create mode 120000 meta-arm/lib/oeqa/selftest/cases/tests/test-parameters.py diff --git a/meta-arm/lib/oeqa/selftest/cases/runfvp.py b/meta-arm/lib/oeqa/selftest/cases/runfvp.py new file mode 100644 index 00000000..5b06ca8e --- /dev/null +++ b/meta-arm/lib/oeqa/selftest/cases/runfvp.py @@ -0,0 +1,51 @@ +import os +import pathlib +import subprocess + +from oeqa.selftest.case import OESelftestTestCase + +runfvp = pathlib.Path(__file__).parents[5] / "scripts" / "runfvp" +testdir = pathlib.Path(__file__).parent / "tests" + +class RunFVPTests(OESelftestTestCase): + def setUpLocal(self): + self.assertTrue(runfvp.exists()) + + def run_fvp(self, *args, should_succeed=True): + """ + Call runfvp passing any arguments. If check is True verify return stdout + on exit code 0 or fail the test, otherwise return the CompletedProcess + instance. + """ + # Put the test directory in PATH so that any mock FVPs are found first + newenv = {"PATH": str(testdir) + ":" + os.environ["PATH"]} + cli = [runfvp,] + list(args) + print(f"Calling {cli}") + ret = subprocess.run(cli, env=newenv, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True) + if should_succeed: + self.assertEqual(ret.returncode, 0, f"runfvp exit {ret.returncode}, output: {ret.stdout}") + return ret.stdout + else: + self.assertNotEqual(ret.returncode, 0, f"runfvp exit {ret.returncode}, output: {ret.stdout}") + return ret.stdout + + def test_help(self): + output = self.run_fvp("--help") + self.assertIn("Run images in a FVP", output) + + def test_bad_options(self): + self.run_fvp("--this-is-an-invalid-option", should_succeed=False) + + def test_run_auto_tests(self): + newenv = {"PATH": str(testdir) + ":" + os.environ["PATH"]} + + cases = list(testdir.glob("auto-*.json")) + if not cases: + self.fail("No tests found") + for case in cases: + with self.subTest(case=case.stem): + self.run_fvp(case) + + def test_fvp_options(self): + # test-parameter sets one argument, add another manually + self.run_fvp(testdir / "test-parameter.json", "--", "--parameter", "board.dog=woof") diff --git a/meta-arm/lib/oeqa/selftest/cases/tests/auto-basic.json b/meta-arm/lib/oeqa/selftest/cases/tests/auto-basic.json new file mode 100644 index 00000000..476eb572 --- /dev/null +++ b/meta-arm/lib/oeqa/selftest/cases/tests/auto-basic.json @@ -0,0 +1,3 @@ +{ + "exe": "auto-basic.sh" +} diff --git a/meta-arm/lib/oeqa/selftest/cases/tests/auto-basic.sh b/meta-arm/lib/oeqa/selftest/cases/tests/auto-basic.sh new file mode 100755 index 00000000..ea9abac1 --- /dev/null +++ b/meta-arm/lib/oeqa/selftest/cases/tests/auto-basic.sh @@ -0,0 +1,11 @@ +#! /bin/sh + +set -e -u + +if [ $# = 0 ]; then + echo No arguments as expected + exit 0 +else + echo Unexpected arguments: $* + exit 1 +fi diff --git a/meta-arm/lib/oeqa/selftest/cases/tests/auto-parameters.json b/meta-arm/lib/oeqa/selftest/cases/tests/auto-parameters.json new file mode 100644 index 00000000..0c7d4ef9 --- /dev/null +++ b/meta-arm/lib/oeqa/selftest/cases/tests/auto-parameters.json @@ -0,0 +1,7 @@ +{ + "exe": "test-parameters.py", + "parameters": { + "board.cow": "moo", + "board.dog": "woof" + } +} diff --git a/meta-arm/lib/oeqa/selftest/cases/tests/mock-fvp.py b/meta-arm/lib/oeqa/selftest/cases/tests/mock-fvp.py new file mode 100755 index 00000000..2213c9f0 --- /dev/null +++ b/meta-arm/lib/oeqa/selftest/cases/tests/mock-fvp.py @@ -0,0 +1,22 @@ +#! /usr/bin/env python3 + +import argparse +import sys + +def do_test_parameters(args): + if not args.parameter or set(args.parameter) != set(("board.cow=moo", "board.dog=woof")): + print(f"Unexpected arguments: {args}") + sys.exit(1) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("-C", "--parameter", action="append") + args = parser.parse_args() + + function = "do_" + parser.prog.replace("-", "_").replace(".py", "") + if function in locals(): + locals()[function](args) + else: + print(f"Unknown mock mode {parser.prog}") + sys.exit(1) diff --git a/meta-arm/lib/oeqa/selftest/cases/tests/test-parameter.json b/meta-arm/lib/oeqa/selftest/cases/tests/test-parameter.json new file mode 100644 index 00000000..9b565f27 --- /dev/null +++ b/meta-arm/lib/oeqa/selftest/cases/tests/test-parameter.json @@ -0,0 +1,6 @@ +{ + "exe": "test-parameters.py", + "parameters": { + "board.cow": "moo" + } +} diff --git a/meta-arm/lib/oeqa/selftest/cases/tests/test-parameters.py b/meta-arm/lib/oeqa/selftest/cases/tests/test-parameters.py new file mode 120000 index 00000000..c734eeca --- /dev/null +++ b/meta-arm/lib/oeqa/selftest/cases/tests/test-parameters.py @@ -0,0 +1 @@ +mock-fvp.py \ No newline at end of file From patchwork Thu Mar 31 18:31:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E883DC4332F for ; Thu, 31 Mar 2022 18:31:23 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.750.1648751482077540218 for ; Thu, 31 Mar 2022 11:31:23 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CAD70139F for ; Thu, 31 Mar 2022 11:31:22 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7CEEC3F718 for ; Thu, 31 Mar 2022 11:31:22 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 5/7] CI: clean up assignments in base.yml Date: Thu, 31 Mar 2022 19:31:15 +0100 Message-Id: <20220331183117.22389-5-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331183117.22389-1-ross.burton@arm.com> References: <20220331183117.22389-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:31:23 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/3221 Split the base local.conf configuration into base (absolutely needed) and setup (typical configuration). This is needed as oe-selftest needs a minimal configuration to execute in, for example doesn't inherit rm_work. Signed-off-by: Ross Burton --- ci/base.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ci/base.yml b/ci/base.yml index 9a59de79..eb5637d4 100644 --- a/ci/base.yml +++ b/ci/base.yml @@ -28,15 +28,15 @@ local_conf_header: base: | BB_SERVER_TIMEOUT = "60" CONF_VERSION = "2" - PACKAGE_CLASSES = "package_ipk" + BB_NUMBER_THREADS = "16" + PARALLEL_MAKE = "-j16" LICENSE_FLAGS_ACCEPTED += "armcompiler Arm-FVP-EULA" + setup: | + PACKAGE_CLASSES = "package_ipk" PACKAGECONFIG:remove:pn-qemu-system-native = "gtk+ sdl" EXTRA_IMAGE_FEATURES:append = " debug-tweaks" - BB_NUMBER_THREADS = "16" - PARALLEL_MAKE = "-j16" - INHERIT += "rm_work" PACKAGECONFIG:append:pn-perf = " coresight" - noptest: | + INHERIT += "rm_work" DISTRO_FEATURES:remove = "ptest" kvm: | QEMU_USE_KVM = "" From patchwork Thu Mar 31 18:31:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8D3CC433EF for ; Thu, 31 Mar 2022 18:31:27 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web10.718.1648751483738155451 for ; Thu, 31 Mar 2022 11:31:23 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5CFCF139F for ; Thu, 31 Mar 2022 11:31:23 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0F1423F718 for ; Thu, 31 Mar 2022 11:31:22 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 6/7] CI: add basic selftest job Date: Thu, 31 Mar 2022 19:31:16 +0100 Message-Id: <20220331183117.22389-6-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331183117.22389-1-ross.burton@arm.com> References: <20220331183117.22389-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:31:27 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/3222 Currently this just executes the runfvp tests, but we can extend it over time. Signed-off-by: Ross Burton --- .gitlab-ci.yml | 8 +++++++- ci/selftest.yml | 7 +++++++ 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100644 ci/selftest.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6d24e0d9..21fcd43f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -25,7 +25,7 @@ stages: - mkdir --verbose --parents $KAS_WORK_DIR $KAS_REPO_REF_DIR $SSTATE_DIR $DL_DIR $TOOLCHAIN_DIR $TOOLCHAIN_LINK_DIR # Must do this here, as it's the only way to make sure the toolchain is installed on the same builder - ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR $TOOLCHAIN_LINK_DIR - - sudo apt update && sudo apt install telnet -y + - sudo apt update && sudo apt install --yes telnet python3-subunit # Generalised fragment to do a Kas build .build: @@ -201,6 +201,12 @@ tc1: toolchains: extends: .build +selftest: + extends: .setup + script: + - KASFILES=./ci/qemuarm64.yml:./ci/selftest.yml + - kas shell --update --force-checkout $KASFILES -c 'oe-selftest --num-processes 1 --run-tests runfvp' + # Validate layers are Yocto Project Compatible check-layers: extends: .setup diff --git a/ci/selftest.yml b/ci/selftest.yml new file mode 100644 index 00000000..9a587354 --- /dev/null +++ b/ci/selftest.yml @@ -0,0 +1,7 @@ +header: + version: 11 + +local_conf_header: + setup: | + BB_LOGCONFIG = "" + SANITY_TESTED_DISTROS = "" From patchwork Thu Mar 31 18:31:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8F45C433F5 for ; Thu, 31 Mar 2022 18:31:24 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web10.719.1648751484275227718 for ; Thu, 31 Mar 2022 11:31:24 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E354813D5 for ; Thu, 31 Mar 2022 11:31:23 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 955873F718 for ; Thu, 31 Mar 2022 11:31:23 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 7/7] arm/oeqa: add no-op runtime test Date: Thu, 31 Mar 2022 19:31:17 +0100 Message-Id: <20220331183117.22389-7-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331183117.22389-1-ross.burton@arm.com> References: <20220331183117.22389-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:31:24 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/3223 Machines which don't have working network stacks in a FVP can still run a useful testimage, because it will validate that the machine will boot to a login prompt on the console. However, we can't set TEST_SUITES to "" because testimage assumes that was a mistake, so add a no-op test case for use in these situations. Signed-off-by: Ross Burton --- meta-arm/lib/oeqa/runtime/cases/noop.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 meta-arm/lib/oeqa/runtime/cases/noop.py diff --git a/meta-arm/lib/oeqa/runtime/cases/noop.py b/meta-arm/lib/oeqa/runtime/cases/noop.py new file mode 100644 index 00000000..b5fba7c5 --- /dev/null +++ b/meta-arm/lib/oeqa/runtime/cases/noop.py @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: MIT + +from oeqa.runtime.case import OERuntimeTestCase + +class NoopTest(OERuntimeTestCase): + """ + This is a test case which does nothing. Useful when you want to use + testimage to verify that an image boots, but you don't have networking so + none of the existing test cases are suitable. + """ + def test_no_op(self): + return