| Message ID | 20250916110116.1227-1-barne.carstensen@danfoss.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | test-manual: update runtime-testing Exporting Tests section | expand |
Hi, On Tue Sep 16, 2025 at 1:01 PM CEST, Barne Carstensen via lists.yoctoproject.org wrote: > Signed-off-by: Barne Carstensen <barne.carstensen@danfoss.com> > --- > documentation/test-manual/runtime-testing.rst | 19 ++++++++++++------- > 1 file changed, 12 insertions(+), 7 deletions(-) > > diff --git a/documentation/test-manual/runtime-testing.rst b/documentation/test-manual/runtime-testing.rst > index 557e0530b..909448ab2 100644 > --- a/documentation/test-manual/runtime-testing.rst > +++ b/documentation/test-manual/runtime-testing.rst > @@ -403,7 +403,7 @@ defined in :term:`TEST_SUITES`. > If your image is already built, make sure the following are set in your > ``local.conf`` file:: > > - INHERIT += "testexport" > + IMAGE_CLASSES += "testexport" > TEST_TARGET_IP = "IP-address-for-the-test-target" > TEST_SERVER_IP = "IP-address-for-the-test-server" > > @@ -413,18 +413,23 @@ following BitBake command form:: > $ bitbake image -c testexport > > Exporting the tests places them in the :term:`Build Directory` in > -``tmp/testexport/``\ image, which is controlled by the :term:`TEST_EXPORT_DIR` > +``tmp/testimage/``\ image, which is controlled by the :term:`TEST_EXPORT_DIR` > variable. > > You can now run the tests outside of the build environment:: > > - $ cd tmp/testexport/image > - $ ./runexported.py testdata.json > + $ cd tmp/testimage/image > + $ ./oe-test runtime > + > +.. note:: > + > + You might need to run the image under QEMU or deploy it to your > + hardware before you can run the tests. > > Here is a complete example that shows IP addresses and uses the > ``core-image-sato`` image:: > > - INHERIT += "testexport" > + IMAGE_CLASSES += "testexport" > TEST_TARGET_IP = "192.168.7.2" > TEST_SERVER_IP = "192.168.7.1" > > @@ -435,8 +440,8 @@ Use BitBake to export the tests:: > Run the tests outside of > the build environment using the following:: > > - $ cd tmp/testexport/core-image-sato > - $ ./runexported.py testdata.json > + $ cd tmp/testimage/core-image-sato > + $ ./oe-test runtime > > Writing New Tests > ================= I've tested these changes and it looks like this applies since ed4238487c81 ("testexport: Fix to work as an image class"). So can be backported to walnascar and scarthgap. I will include this in the commit message, as I think the patch itself is correct. Next time, can you please include a commit message body describing the change and ideally referencing the commit that implied this change? I've tested this on a core-image-minimal/qemuarm64 build and was able to run some tests with your modifications. So: Tested-by: Antonin Godard <antonin.godard@bootlin.com> Reviewed-by: Antonin Godard <antonin.godard@bootlin.com> I just had issues, I wonder if you also encountered them: File ".../tmp/testimage/core-image-minimal/meta/lib/oeqa/runtime/cases/storage.py", line 129, in setUpClass self.test_dir = os.path.join(self.mount_point, "oeqa") ^^ NameError: name 'os' is not defined I have the feeling that this works when using `bitbake -c testimage <image>` directly, because in bitbake the os module is available by default, and OE-Core test were written with this in mind? But in an exported environment, you obviously don't have this module imported by default, like any other Python program (even after having sourced the environment). In that case this would maybe mean that: - we should import os in these files. - or we consider that these tests were made for OECore testing with testimage only. Thanks! Antonin
Hello, Thank you for reviewing and testing. Sure, I will add more information to the commit message next time. With the "commit that implied this change" you mean the commit in oe-core which switched from runexported.py to oe-test? I did not notice the error, btw. Barne Classified as Business -----Original Message----- From: Antonin Godard <antonin.godard@bootlin.com> Sent: Mittwoch, 17. September 2025 16:04 To: Barne Carstensen <barne.carstensen@danfoss.com>; docs@lists.yoctoproject.org Subject: Re: [docs] [PATCH] test-manual: update runtime-testing Exporting Tests section [You don't often get email from antonin.godard@bootlin.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi, On Tue Sep 16, 2025 at 1:01 PM CEST, Barne Carstensen via lists.yoctoproject.org wrote: > Signed-off-by: Barne Carstensen <barne.carstensen@danfoss.com> > --- > documentation/test-manual/runtime-testing.rst | 19 > ++++++++++++------- > 1 file changed, 12 insertions(+), 7 deletions(-) > > diff --git a/documentation/test-manual/runtime-testing.rst > b/documentation/test-manual/runtime-testing.rst > index 557e0530b..909448ab2 100644 > --- a/documentation/test-manual/runtime-testing.rst > +++ b/documentation/test-manual/runtime-testing.rst > @@ -403,7 +403,7 @@ defined in :term:`TEST_SUITES`. > If your image is already built, make sure the following are set in > your ``local.conf`` file:: > > - INHERIT += "testexport" > + IMAGE_CLASSES += "testexport" > TEST_TARGET_IP = "IP-address-for-the-test-target" > TEST_SERVER_IP = "IP-address-for-the-test-server" > > @@ -413,18 +413,23 @@ following BitBake command form:: > $ bitbake image -c testexport > > Exporting the tests places them in the :term:`Build Directory` in > -``tmp/testexport/``\ image, which is controlled by the > :term:`TEST_EXPORT_DIR` > +``tmp/testimage/``\ image, which is controlled by the > +:term:`TEST_EXPORT_DIR` > variable. > > You can now run the tests outside of the build environment:: > > - $ cd tmp/testexport/image > - $ ./runexported.py testdata.json > + $ cd tmp/testimage/image > + $ ./oe-test runtime > + > +.. note:: > + > + You might need to run the image under QEMU or deploy it to your > + hardware before you can run the tests. > > Here is a complete example that shows IP addresses and uses the > ``core-image-sato`` image:: > > - INHERIT += "testexport" > + IMAGE_CLASSES += "testexport" > TEST_TARGET_IP = "192.168.7.2" > TEST_SERVER_IP = "192.168.7.1" > > @@ -435,8 +440,8 @@ Use BitBake to export the tests:: > Run the tests outside of > the build environment using the following:: > > - $ cd tmp/testexport/core-image-sato > - $ ./runexported.py testdata.json > + $ cd tmp/testimage/core-image-sato > + $ ./oe-test runtime > > Writing New Tests > ================= I've tested these changes and it looks like this applies since ed4238487c81 ("testexport: Fix to work as an image class"). So can be backported to walnascar and scarthgap. I will include this in the commit message, as I think the patch itself is correct. Next time, can you please include a commit message body describing the change and ideally referencing the commit that implied this change? I've tested this on a core-image-minimal/qemuarm64 build and was able to run some tests with your modifications. So: Tested-by: Antonin Godard <antonin.godard@bootlin.com> Reviewed-by: Antonin Godard <antonin.godard@bootlin.com> I just had issues, I wonder if you also encountered them: File ".../tmp/testimage/core-image-minimal/meta/lib/oeqa/runtime/cases/storage.py", line 129, in setUpClass self.test_dir = os.path.join(self.mount_point, "oeqa") ^^ NameError: name 'os' is not defined I have the feeling that this works when using `bitbake -c testimage <image>` directly, because in bitbake the os module is available by default, and OE-Core test were written with this in mind? But in an exported environment, you obviously don't have this module imported by default, like any other Python program (even after having sourced the environment). In that case this would maybe mean that: - we should import os in these files. - or we consider that these tests were made for OECore testing with testimage only. Thanks! Antonin -- Antonin Godard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com/
On Tue, 16 Sep 2025 11:01:22 +0000, Barne Carstensen wrote:
>
Applied, thanks!
[1/1] test-manual: update runtime-testing Exporting Tests section
commit: 2caa8e581feaf3640bea68108f9a02583b17b21b
Best regards,
diff --git a/documentation/test-manual/runtime-testing.rst b/documentation/test-manual/runtime-testing.rst index 557e0530b..909448ab2 100644 --- a/documentation/test-manual/runtime-testing.rst +++ b/documentation/test-manual/runtime-testing.rst @@ -403,7 +403,7 @@ defined in :term:`TEST_SUITES`. If your image is already built, make sure the following are set in your ``local.conf`` file:: - INHERIT += "testexport" + IMAGE_CLASSES += "testexport" TEST_TARGET_IP = "IP-address-for-the-test-target" TEST_SERVER_IP = "IP-address-for-the-test-server" @@ -413,18 +413,23 @@ following BitBake command form:: $ bitbake image -c testexport Exporting the tests places them in the :term:`Build Directory` in -``tmp/testexport/``\ image, which is controlled by the :term:`TEST_EXPORT_DIR` +``tmp/testimage/``\ image, which is controlled by the :term:`TEST_EXPORT_DIR` variable. You can now run the tests outside of the build environment:: - $ cd tmp/testexport/image - $ ./runexported.py testdata.json + $ cd tmp/testimage/image + $ ./oe-test runtime + +.. note:: + + You might need to run the image under QEMU or deploy it to your + hardware before you can run the tests. Here is a complete example that shows IP addresses and uses the ``core-image-sato`` image:: - INHERIT += "testexport" + IMAGE_CLASSES += "testexport" TEST_TARGET_IP = "192.168.7.2" TEST_SERVER_IP = "192.168.7.1" @@ -435,8 +440,8 @@ Use BitBake to export the tests:: Run the tests outside of the build environment using the following:: - $ cd tmp/testexport/core-image-sato - $ ./runexported.py testdata.json + $ cd tmp/testimage/core-image-sato + $ ./oe-test runtime Writing New Tests =================
Signed-off-by: Barne Carstensen <barne.carstensen@danfoss.com> --- documentation/test-manual/runtime-testing.rst | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)