From patchwork Thu Dec 26 14:38:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonin Godard X-Patchwork-Id: 54699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FC03E7718F for ; Thu, 26 Dec 2024 14:38:32 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by mx.groups.io with SMTP id smtpd.web11.22231.1735223910996320147 for ; Thu, 26 Dec 2024 06:38:31 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=ED5MPYra; spf=pass (domain: bootlin.com, ip: 217.70.183.197, mailfrom: antonin.godard@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 2B7BD1C0008; Thu, 26 Dec 2024 14:38:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1735223909; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+1BH4HLusxvq8FDLmqJLrYBlFLFqRjrAFl8tAz7ZNqk=; b=ED5MPYra7d9o9VNUYejUh6uBY4ocbOIFLpfbMALa+Edt7WrI2aR4nDTiCQmUlAaaxKLJKW yN3uxxdc6dAv5QdoEtIFXb+upt0YBkzTcInUEiyN1Yn6fwWVlvqjQbm8g+gBCsDbpvfRb9 pIXQp/Ti4kwCwt1wPq6MavgGgSqiEEP9xcVMxUQTuntocMQeWsYZ6xZWBuC6vHe+7UofBi 0Ya0gnbde15z05ZrrMv/VZKjSQ1uHeHHxqqc/RQHJbLNEjFmjMgxhEARiaF5aFxllqfo5v N+1e8mB10du3QkzLbUiZesJTmX/bmWtjslDbXaCacxvrV4c0HKGWhoJviWxnlg== From: Antonin Godard Date: Thu, 26 Dec 2024 15:38:07 +0100 Subject: [PATCH 1/2] ref-manual/packages: move ptest section to the test-manual MIME-Version: 1.0 Message-Id: <20241226-move-ptest-to-test-manual-v1-1-ef16b91c8971@bootlin.com> References: <20241226-move-ptest-to-test-manual-v1-0-ef16b91c8971@bootlin.com> In-Reply-To: <20241226-move-ptest-to-test-manual-v1-0-ef16b91c8971@bootlin.com> To: docs@lists.yoctoproject.org Cc: Thomas Petazzoni , Antonin Godard , Yoann Congal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=openpgp-sha256; l=11902; i=antonin.godard@bootlin.com; h=from:subject:message-id; bh=fIcI3edttlsx5xZxWTndNn+fuu9sIkJqoJh/8y3Su/E=; b=owEBbQKS/ZANAwAIAdGAQUApo6g2AcsmYgBnbWpkvs7Ct4BBaFjokhCT0yz+VcXGxFkvCtckY 8j8u3xAumOJAjMEAAEIAB0WIQSGSHJRiN1AG7mg0//RgEFAKaOoNgUCZ21qZAAKCRDRgEFAKaOo NtEwEACw4B1IdFCbeDf0yA7VluILOY+9OAoY7YpwCGMl3sYetv1yvuVVhBWyNaRAJbJHKyMrPm9 WgnzvI8gicN7/Yzzqgx+UUqvASWFkprSoBf0EBsUYvTC7DX0fVWGG9S36fuwfC+2Iaux+7FkxBx S08+Y0gstlgnCtlN8kMMdqwuTNaznSuc0KibK+wd23cSC1DgGwHjOz6hHG0tIazevdt97aBmtoQ doIx29BeMICzzO2XjY2D9EnxBnFVl706QcgPTc9H7Da6ry4UcsiO6b9kgEYk42BLXqDwvhuJ5YD azL1kx+FniZLMzCE7Ti+h6H0Hr20Hu/N92AG+n9gJIht56BeG+cykWkWwVlAIZmx8y01qzdGoAY Wcedev9pUNgybBRd2jpd7uIjyfgqO+uH5tPfwQKZnXVJrB1Nl4oVQk8BvfWqyogMlO6mNpVfUWo zoxdC4l8uvR+Z1i2RQjCizIOGybsabvR9A8jCBBOuh1Ldv4fTjD5M69v40YZ1MBDTB70BoUBo6K 0Rg10UEARMwW5496RxXxILvKvwwp4C3mz+wgWgqF+L/y+kcuDSiD+E+OX9MNNWAw3HygsM3vn2Z RB1F1oKBaNFfRO6qGtNfQ7O506HcliGVcNuGp0+C5zthuMwlkXpW4KFWCtydp14x7Gj3iLfZBeM r8oP/ocY5kW1YZg== X-Developer-Key: i=antonin.godard@bootlin.com; a=openpgp; fpr=8648725188DD401BB9A0D3FFD180414029A3A836 X-GND-Sasl: antonin.godard@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 Dec 2024 14:38:32 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/6041 [ YOCTO #15106 ] It makes more sense to document ptests in the test-manual. Since ptests are still related to packages, keep a link to ptests from packages.rst to the test-manual. Reported-by: Yoann Congal Signed-off-by: Antonin Godard --- documentation/dev-manual/packages.rst | 110 +------------------------------- documentation/test-manual/index.rst | 1 + documentation/test-manual/intro.rst | 2 +- documentation/test-manual/ptest.rst | 114 ++++++++++++++++++++++++++++++++++ 4 files changed, 118 insertions(+), 109 deletions(-) diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst index 33cf78747d1bc62268811321ad7a40397268e13f..75964598c8222b7390a96873bc4a3bf37bf8c45b 100644 --- a/documentation/dev-manual/packages.rst +++ b/documentation/dev-manual/packages.rst @@ -882,114 +882,8 @@ related to signed package feeds are available: Testing Packages With ptest =========================== -A Package Test (ptest) runs tests against packages built by the -OpenEmbedded build system on the target machine. A ptest contains at -least two items: the actual test, and a shell script (``run-ptest``) -that starts the test. The shell script that starts the test must not -contain the actual test --- the script only starts the test. On the other -hand, the test can be anything from a simple shell script that runs a -binary and checks the output to an elaborate system of test binaries and -data files. - -The test generates output in the format used by Automake:: - - result: testname - -where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and -the testname can be any identifying string. - -For a list of Yocto Project recipes that are already enabled with ptest, -see the :yocto_wiki:`Ptest ` wiki page. - -.. note:: - - A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` - class. - -Adding ptest to Your Build --------------------------- - -To add package testing to your build, add the :term:`DISTRO_FEATURES` and -:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which -is found in the :term:`Build Directory`:: - - DISTRO_FEATURES:append = " ptest" - EXTRA_IMAGE_FEATURES += "ptest-pkgs" - -Once your build is complete, the ptest files are installed into the -``/usr/lib/package/ptest`` directory within the image, where ``package`` -is the name of the package. - -Running ptest -------------- - -The ``ptest-runner`` package installs a shell script that loops through -all installed ptest test suites and runs them in sequence. Consequently, -you might want to add this package to your image. - -Getting Your Package Ready --------------------------- - -In order to enable a recipe to run installed ptests on target hardware, -you need to prepare the recipes that build the packages you want to -test. Here is what you have to do for each recipe: - -- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* - Include the following line in each recipe:: - - inherit ptest - -- *Create run-ptest:* This script starts your test. Locate the - script where you will refer to it using - :term:`SRC_URI`. Here is an - example that starts a test for ``dbus``:: - - #!/bin/sh - cd test - make -k runtest-TESTS - -- *Ensure dependencies are met:* If the test adds build or runtime - dependencies that normally do not exist for the package (such as - requiring "make" to run the test suite), use the - :term:`DEPENDS` and - :term:`RDEPENDS` variables in - your recipe in order for the package to meet the dependencies. Here - is an example where the package has a runtime dependency on "make":: - - RDEPENDS:${PN}-ptest += "make" - -- *Add a function to build the test suite:* Not many packages support - cross-compilation of their test suites. Consequently, you usually - need to add a cross-compilation function to the package. - - Many packages based on Automake compile and run the test suite by - using a single command such as ``make check``. However, the host - ``make check`` builds and runs on the same computer, while - cross-compiling requires that the package is built on the host but - executed for the target architecture (though often, as in the case - for ptest, the execution occurs on the host). The built version of - Automake that ships with the Yocto Project includes a patch that - separates building and execution. Consequently, packages that use the - unaltered, patched version of ``make check`` automatically - cross-compiles. - - Regardless, you still must add a ``do_compile_ptest`` function to - build the test suite. Add a function similar to the following to your - recipe:: - - do_compile_ptest() { - oe_runmake buildtest-TESTS - } - -- *Ensure special configurations are set:* If the package requires - special configurations prior to compiling the test code, you must - insert a ``do_configure_ptest`` function into the recipe. - -- *Install the test suite:* The :ref:`ref-classes-ptest` class - automatically copies the file ``run-ptest`` to the target and then runs make - ``install-ptest`` to run the tests. If this is not enough, you need - to create a ``do_install_ptest`` function and make sure it gets - called after the "make install-ptest" completes. +See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the +Yocto Project Test Environment Manual. Creating Node Package Manager (NPM) Packages ============================================ diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst index 86a2f436ea04e8f6038037e79898218b058bde4a..ad71f379106261af32dc925d700ed36da43d234b 100644 --- a/documentation/test-manual/index.rst +++ b/documentation/test-manual/index.rst @@ -12,6 +12,7 @@ Yocto Project Test Environment Manual intro test-process + ptest understand-autobuilder reproducible-builds yocto-project-compatible diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst index 882ca16486722675264b43887c425880c80f6af7..252d3ea731c39709b8fbe64d281722c172a282a9 100644 --- a/documentation/test-manual/intro.rst +++ b/documentation/test-manual/intro.rst @@ -140,7 +140,7 @@ the following types of tests: - *Package Testing:* A Package Test (ptest) runs tests against packages built by the OpenEmbedded build system on the target machine. See the :ref:`Testing Packages With - ptest ` section + ptest ` section in the Yocto Project Development Tasks Manual and the ":yocto_wiki:`Ptest `" Wiki page for more information on Ptest. diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst new file mode 100644 index 0000000000000000000000000000000000000000..dea1bad23bd98817a2020030efdad7ce4e4b36c0 --- /dev/null +++ b/documentation/test-manual/ptest.rst @@ -0,0 +1,114 @@ +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK + +*************************** +Testing Packages With ptest +*************************** + +A Package Test (ptest) runs tests against packages built by the +OpenEmbedded build system on the target machine. A ptest contains at +least two items: the actual test, and a shell script (``run-ptest``) +that starts the test. The shell script that starts the test must not +contain the actual test --- the script only starts the test. On the other +hand, the test can be anything from a simple shell script that runs a +binary and checks the output to an elaborate system of test binaries and +data files. + +The test generates output in the format used by Automake:: + + result: testname + +where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and +the testname can be any identifying string. + +For a list of Yocto Project recipes that are already enabled with ptest, +see the :yocto_wiki:`Ptest ` wiki page. + +.. note:: + + A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` + class. + +Adding ptest to Your Build +========================== + +To add package testing to your build, add the :term:`DISTRO_FEATURES` and +:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which +is found in the :term:`Build Directory`:: + + DISTRO_FEATURES:append = " ptest" + EXTRA_IMAGE_FEATURES += "ptest-pkgs" + +Once your build is complete, the ptest files are installed into the +``/usr/lib/package/ptest`` directory within the image, where ``package`` +is the name of the package. + +Running ptest +============= + +The ``ptest-runner`` package installs a shell script that loops through +all installed ptest test suites and runs them in sequence. Consequently, +you might want to add this package to your image. + +Getting Your Package Ready +========================== + +In order to enable a recipe to run installed ptests on target hardware, +you need to prepare the recipes that build the packages you want to +test. Here is what you have to do for each recipe: + +- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* + Include the following line in each recipe:: + + inherit ptest + +- *Create run-ptest:* This script starts your test. Locate the + script where you will refer to it using + :term:`SRC_URI`. Here is an + example that starts a test for ``dbus``:: + + #!/bin/sh + cd test + make -k runtest-TESTS + +- *Ensure dependencies are met:* If the test adds build or runtime + dependencies that normally do not exist for the package (such as + requiring "make" to run the test suite), use the + :term:`DEPENDS` and + :term:`RDEPENDS` variables in + your recipe in order for the package to meet the dependencies. Here + is an example where the package has a runtime dependency on "make":: + + RDEPENDS:${PN}-ptest += "make" + +- *Add a function to build the test suite:* Not many packages support + cross-compilation of their test suites. Consequently, you usually + need to add a cross-compilation function to the package. + + Many packages based on Automake compile and run the test suite by + using a single command such as ``make check``. However, the host + ``make check`` builds and runs on the same computer, while + cross-compiling requires that the package is built on the host but + executed for the target architecture (though often, as in the case + for ptest, the execution occurs on the host). The built version of + Automake that ships with the Yocto Project includes a patch that + separates building and execution. Consequently, packages that use the + unaltered, patched version of ``make check`` automatically + cross-compiles. + + Regardless, you still must add a ``do_compile_ptest`` function to + build the test suite. Add a function similar to the following to your + recipe:: + + do_compile_ptest() { + oe_runmake buildtest-TESTS + } + +- *Ensure special configurations are set:* If the package requires + special configurations prior to compiling the test code, you must + insert a ``do_configure_ptest`` function into the recipe. + +- *Install the test suite:* The :ref:`ref-classes-ptest` class + automatically copies the file ``run-ptest`` to the target and then runs make + ``install-ptest`` to run the tests. If this is not enough, you need + to create a ``do_install_ptest`` function and make sure it gets + called after the "make install-ptest" completes. From patchwork Thu Dec 26 14:38:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonin Godard X-Patchwork-Id: 54698 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79A62E7718E for ; Thu, 26 Dec 2024 14:38:32 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by mx.groups.io with SMTP id smtpd.web11.22230.1735223910915566117 for ; Thu, 26 Dec 2024 06:38:31 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=QSY6FaOY; spf=pass (domain: bootlin.com, ip: 217.70.183.197, mailfrom: antonin.godard@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 6F3851C0009; Thu, 26 Dec 2024 14:38:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1735223909; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pNH0TFFKCbGhgvdEYVV3xfd0ef0VE684AppvDA2RHi8=; b=QSY6FaOYcfYjGKAUjSNqBG26ZuAl5nJqGUkbgE76H5qxTd8dJZDq28pMf91zM3yvrY4yL7 fqHieb5e5z64iTI7wHkzjiNtgR6//Opqi3Qm4PuGYz0Js8UraphMLVg/wAGuD+DKD1wSIG 8Q4gsOIu7UVhh7TEodve0tui0TTw66Os2ZDkStwcmft+23lfYidfJzIJcHi35poVbbcN+s 0K9xp7BkXUght0T2giWGNagQAv8mZt9rGIUBIjkbhsTFYNW7Iow/jdhknDwoXZSaqod7b8 2Mb5RZHyASDmKcWuc0/6RnFvkkY2NiejwMHaRTPEZwAFL/aoROdgS6z6tg85xQ== From: Antonin Godard Date: Thu, 26 Dec 2024 15:38:08 +0100 Subject: [PATCH 2/2] ref-manual: move runtime-testing section to the test-manual MIME-Version: 1.0 Message-Id: <20241226-move-ptest-to-test-manual-v1-2-ef16b91c8971@bootlin.com> References: <20241226-move-ptest-to-test-manual-v1-0-ef16b91c8971@bootlin.com> In-Reply-To: <20241226-move-ptest-to-test-manual-v1-0-ef16b91c8971@bootlin.com> To: docs@lists.yoctoproject.org Cc: Thomas Petazzoni , Antonin Godard X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=openpgp-sha256; l=12230; i=antonin.godard@bootlin.com; h=from:subject:message-id; bh=zs+BGvTgNkIfFf6qSJdkBhT9S0Q6Ji9sv7NElB/R8gc=; b=owEBbQKS/ZANAwAIAdGAQUApo6g2AcsmYgBnbWpkdPqvZaeC/tCqVRNIq0DBaesmP5bNtvZ3g go4mmK3XH+JAjMEAAEIAB0WIQSGSHJRiN1AG7mg0//RgEFAKaOoNgUCZ21qZAAKCRDRgEFAKaOo Nr6eEACn1ueA9aR2s2rYqLl9Eo6dZcDcdujCNyAdO3SgMJ24PNC/rvz6EK+Pf5xyNohHlLLpU9b JzR+/1jtTEtm4AVnDHWLOxYqYXxh249Kyas+WaEC2FOSl3+6n1QD2v4Stnd1NIEniCGlupmzqzS 0XOHyLw/UIYF7tnhY26MqEHbAsCIj7AvAR+oUOAO4R4xGgk+Ki+xKKH7ewFdfO9ew89geaQbjQ8 DBu7uNDk8fNQ5x0X/+M8ulePv8G0IWU3j2h5H7lr9LTiq9UzwQgRYTdLBPgXxu815slIQGtWUTn B8mCYF1aMLdbQ5UwNTd2vskTkh7c6iz5SwIL44TrGpAn4OL6Cwh4ky/C6a//XY+Y7VC5k0bVmhY W5qoBcORZBmAYtnx+a6yley+Y4QoCooF8FjXbFwaqxkeElm4aDbDwjE1Lxz6nNEd7PSlO2Sl7u3 YdDnjDgrJx7UgHH4CDJSfQAdEUUk0k6PatOb9Ul4gqskmuKIFTzWQEDZ7KbQ55ltLmGIY85Lq4d kvj3fxHgE13jkbMvlYWhsaBu5gTeg/jgXSZsepogeAjXu5zOjIaciTLVKkelFDId61eRUJQfwtw /8YyFzQE95g5nURHDb8IyrTOqGAwBm/rrjcrSXoRlaJdYLCsTYN99+OELAhe5qejwLLrbZ5gtrz fYD68P9252AbqQQ== X-Developer-Key: i=antonin.godard@bootlin.com; a=openpgp; fpr=8648725188DD401BB9A0D3FFD180414029A3A836 X-GND-Sasl: antonin.godard@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 Dec 2024 14:38:32 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/6039 In the same fashion as 9e63df6cc2f2d3ecfa86009cdc180dd5ec08726a ("ref-manual/packages: move ptest section to the test-manual"), move the runtime testing section of the development tasks manual to the test environment manual. Add a link to it from the test-manual/intro document. Signed-off-by: Antonin Godard --- documentation/dev-manual/index.rst | 1 - documentation/migration-guides/migration-1.5.rst | 4 ++-- documentation/ref-manual/classes.rst | 4 ++-- documentation/ref-manual/images.rst | 4 ++-- documentation/ref-manual/release-process.rst | 4 ++-- documentation/ref-manual/tasks.rst | 8 ++++---- documentation/ref-manual/variables.rst | 20 ++++++++++---------- documentation/test-manual/index.rst | 1 + documentation/test-manual/intro.rst | 4 +++- .../{dev-manual => test-manual}/runtime-testing.rst | 4 ++-- 10 files changed, 28 insertions(+), 26 deletions(-) diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst index 7afd0d820e94d56cb2145962b9b1a976c7951936..8243c0f4cb2598fedee27d3d29e5d0d5b3d1e107 100644 --- a/documentation/dev-manual/index.rst +++ b/documentation/dev-manual/index.rst @@ -39,7 +39,6 @@ Yocto Project Development Tasks Manual external-scm read-only-rootfs build-quality - runtime-testing debugging licenses security-subjects diff --git a/documentation/migration-guides/migration-1.5.rst b/documentation/migration-guides/migration-1.5.rst index c8f3cbc1655421e43968af23faad08d4ce1bd525..da26cca63d2336ada8acda3ea22dc8d45a47bbc2 100644 --- a/documentation/migration-guides/migration-1.5.rst +++ b/documentation/migration-guides/migration-1.5.rst @@ -248,8 +248,8 @@ A new automated image testing framework has been added through the framework replaces the older ``imagetest-qemu`` framework. You can learn more about performing automated image tests in the -":ref:`dev-manual/runtime-testing:performing automated runtime testing`" -section in the Yocto Project Development Tasks Manual. +":ref:`test-manual/runtime-testing:performing automated runtime testing`" +section in the Yocto Project Test Environment Manual. .. _migration-1.5-build-history: diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index e5fe44052d4b05bf1b2b3acfaf2fd368f66fb261..73c79e592af245a293610e2804d90a57fccdcf08 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst @@ -3231,8 +3231,8 @@ after it is built, you can set :term:`TESTIMAGE_AUTO`:: TESTIMAGE_AUTO = "1" For information on how to enable, run, and create new tests, see the -":ref:`dev-manual/runtime-testing:performing automated runtime testing`" -section in the Yocto Project Development Tasks Manual. +":ref:`test-manual/runtime-testing:performing automated runtime testing`" +section in the Yocto Project Test Environment Manual. .. _ref-classes-testsdk: diff --git a/documentation/ref-manual/images.rst b/documentation/ref-manual/images.rst index c45f9104a949f7470fe11e8e70533c7a2ee8b16e..d6bdc92f0744eae5761bf499b8a0e2bd094f1822 100644 --- a/documentation/ref-manual/images.rst +++ b/documentation/ref-manual/images.rst @@ -119,8 +119,8 @@ Here is a list of supported recipes: deployed to a separate partition so that you can boot into it and use it to deploy a second image to be tested. You can find more information about runtime testing in the - ":ref:`dev-manual/runtime-testing:performing automated runtime testing`" - section in the Yocto Project Development Tasks Manual. + ":ref:`test-manual/runtime-testing:performing automated runtime testing`" + section in the Yocto Project Test Environment Manual. - ``core-image-testmaster-initramfs``: A RAM-based Initial Root Filesystem (:term:`Initramfs`) image tailored for use with the diff --git a/documentation/ref-manual/release-process.rst b/documentation/ref-manual/release-process.rst index 691e7d3ed90c98e43e558809828b5094670ccca3..bde1ffbdb9e74205e330cdbac2a7279b4a1e1140 100644 --- a/documentation/ref-manual/release-process.rst +++ b/documentation/ref-manual/release-process.rst @@ -148,8 +148,8 @@ Additionally, because the test strategies are visible to you as a developer, you can validate your projects. This section overviews the available test infrastructure used in the Yocto Project. For information on how to run available tests on your projects, see the -":ref:`dev-manual/runtime-testing:performing automated runtime testing`" -section in the Yocto Project Development Tasks Manual. +":ref:`test-manual/runtime-testing:performing automated runtime testing`" +section in the Yocto Project Test Environment Manual. The QA/testing infrastructure is woven into the project to the point where core developers take some of it for granted. The infrastructure diff --git a/documentation/ref-manual/tasks.rst b/documentation/ref-manual/tasks.rst index df751d75a34bcdba51d9838a819dc9c1d99f19e6..d2e1f4ce143c6dce2c9110a8c98d1945faf71fea 100644 --- a/documentation/ref-manual/tasks.rst +++ b/documentation/ref-manual/tasks.rst @@ -615,8 +615,8 @@ information on how the root filesystem is created. Boots an image and performs runtime tests within the image. For information on automatically testing images, see the -":ref:`dev-manual/runtime-testing:performing automated runtime testing`" -section in the Yocto Project Development Tasks Manual. +":ref:`test-manual/runtime-testing:performing automated runtime testing`" +section in the Yocto Project Test Environment Manual. .. _ref-tasks-testimage_auto: @@ -628,8 +628,8 @@ after it has been built. This task is enabled when you set :term:`TESTIMAGE_AUTO` equal to "1". For information on automatically testing images, see the -":ref:`dev-manual/runtime-testing:performing automated runtime testing`" -section in the Yocto Project Development Tasks Manual. +":ref:`test-manual/runtime-testing:performing automated runtime testing`" +section in the Yocto Project Test Environment Manual. Kernel-Related Tasks ==================== diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index e8db89f8c9642b1e36c574e1369093a7a2ce30a4..96a6be7d6ae23d4565d1bad2467e57dd7b1f58cd 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst @@ -9204,8 +9204,8 @@ system and gives an overview of their function and contents. file. For more information on testing images, see the - ":ref:`dev-manual/runtime-testing:performing automated runtime testing`" - section in the Yocto Project Development Tasks Manual. + ":ref:`test-manual/runtime-testing:performing automated runtime testing`" + section in the Yocto Project Test Environment Manual. :term:`TEST_SERIALCONTROL_CMD` For automated hardware testing, specifies the command to use to @@ -9276,8 +9276,8 @@ system and gives an overview of their function and contents. TEST_SUITES = "test_A test_B" For more information on testing images, see the - ":ref:`dev-manual/runtime-testing:performing automated runtime testing`" - section in the Yocto Project Development Tasks Manual. + ":ref:`test-manual/runtime-testing:performing automated runtime testing`" + section in the Yocto Project Test Environment Manual. :term:`TEST_TARGET` Specifies the target controller to use when running tests against a @@ -9295,8 +9295,8 @@ system and gives an overview of their function and contents. You can provide the following arguments with :term:`TEST_TARGET`: - *"qemu":* Boots a QEMU image and runs the tests. See the - ":ref:`dev-manual/runtime-testing:enabling runtime tests on qemu`" section - in the Yocto Project Development Tasks Manual for more + ":ref:`test-manual/runtime-testing:enabling runtime tests on qemu`" section + in the Yocto Project Test Environment Manual for more information. - *"simpleremote":* Runs the tests on target hardware that is @@ -9311,8 +9311,8 @@ system and gives an overview of their function and contents. ``meta/lib/oeqa/controllers/simpleremote.py``. For information on running tests on hardware, see the - ":ref:`dev-manual/runtime-testing:enabling runtime tests on hardware`" - section in the Yocto Project Development Tasks Manual. + ":ref:`test-manual/runtime-testing:enabling runtime tests on hardware`" + section in the Yocto Project Test Environment Manual. :term:`TEST_TARGET_IP` The IP address of your hardware under test. The :term:`TEST_TARGET_IP` @@ -9348,8 +9348,8 @@ system and gives an overview of their function and contents. For more information on enabling, running, and writing these tests, see the - ":ref:`dev-manual/runtime-testing:performing automated runtime testing`" - section in the Yocto Project Development Tasks Manual and the + ":ref:`test-manual/runtime-testing:performing automated runtime testing`" + section in the Yocto Project Test Environment Manual and the ":ref:`ref-classes-testimage`" section. :term:`TESTIMAGE_FAILED_QA_ARTIFACTS` diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst index ad71f379106261af32dc925d700ed36da43d234b..d365d337ea5ba71a71707aee996444cb7d155ddc 100644 --- a/documentation/test-manual/index.rst +++ b/documentation/test-manual/index.rst @@ -13,6 +13,7 @@ Yocto Project Test Environment Manual intro test-process ptest + runtime-testing understand-autobuilder reproducible-builds yocto-project-compatible diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst index 252d3ea731c39709b8fbe64d281722c172a282a9..1a8e938a2ea23e1b441e0b30ac4fa69229dafac6 100644 --- a/documentation/test-manual/intro.rst +++ b/documentation/test-manual/intro.rst @@ -130,7 +130,9 @@ the following types of tests: $ bitbake image -c testimage The tests use the :ref:`ref-classes-testimage` - class and the :ref:`ref-tasks-testimage` task. + class and the :ref:`ref-tasks-testimage` task. See the + :ref:`test-manual/runtime-testing:Performing Automated Runtime Testing` + section of the Yocto Project Test Environment Manual for more information. - *Layer Testing:* The Autobuilder has the possibility to test whether specific layers work with the test of the system. The layers tested diff --git a/documentation/dev-manual/runtime-testing.rst b/documentation/test-manual/runtime-testing.rst similarity index 99% rename from documentation/dev-manual/runtime-testing.rst rename to documentation/test-manual/runtime-testing.rst index 7a2b42f25af2cd6e3488d706276d52f0cc6ebae9..5cd4fb2ccf9bb3e12e31c8f754b1d1280a8930b7 100644 --- a/documentation/dev-manual/runtime-testing.rst +++ b/documentation/test-manual/runtime-testing.rst @@ -153,7 +153,7 @@ options are available: If you choose "SystemdbootTarget", there are additional requirements and considerations. See the - ":ref:`dev-manual/runtime-testing:selecting systemdboottarget`" section, which + ":ref:`test-manual/runtime-testing:selecting systemdboottarget`" section, which follows, for more information. - *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying @@ -179,7 +179,7 @@ Selecting SystemdbootTarget If you did not set :term:`TEST_TARGET` to "SystemdbootTarget", then you do not need any information in this section. You can skip down to the -":ref:`dev-manual/runtime-testing:running tests`" section. +":ref:`test-manual/runtime-testing:running tests`" section. If you did set :term:`TEST_TARGET` to "SystemdbootTarget", you also need to perform a one-time setup of your controller image by doing the following: