diff mbox series

[v2,6/6] oeqa: ltp - add LTP_WORKERS to allow parallelization

Message ID 20260327122758.1851989-6-daniel.turull@ericsson.com
State Under Review
Headers show
Series [v2,1/6] python3-kirk: Add version 4.0.0 | expand

Commit Message

Daniel Turull March 27, 2026, 12:27 p.m. UTC 
From: Daniel Turull <daniel.turull@ericsson.com>

Add new configurable variable to execute multiple workers in parallel.
kirk adds the option that was not available in runltp.

Tested in qemux86-64 with configuration:

IMAGE_CLASSES += "testimage"
CORE_IMAGE_EXTRA_INSTALL += "ltp openssh"
TEST_SUITES = "ping ssh ltp"
QB_MEM = "32768"
QB_CPU_KVM = "-cpu host -smp 16"
QB_SMP = "-smp 16"
LTP_WORKERS = "16"
IMAGE_ROOTFS_EXTRA_SPACE = "2097152"

bitbake core-image-minimal
bitbake core-image-minimal -c testimage

Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
 meta/lib/oeqa/runtime/cases/ltp.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Comments

Richard Purdie March 27, 2026, 3:37 p.m. UTC | #1 
On Fri, 2026-03-27 at 13:27 +0100, daniel.turull@ericsson.com wrote:
> From: Daniel Turull <daniel.turull@ericsson.com>
> 
> Add new configurable variable to execute multiple workers in parallel.
> kirk adds the option that was not available in runltp.
> 
> Tested in qemux86-64 with configuration:
> 
> IMAGE_CLASSES += "testimage"
> CORE_IMAGE_EXTRA_INSTALL += "ltp openssh"
> TEST_SUITES = "ping ssh ltp"
> QB_MEM = "32768"
> QB_CPU_KVM = "-cpu host -smp 16"
> QB_SMP = "-smp 16"
> LTP_WORKERS = "16"
> IMAGE_ROOTFS_EXTRA_SPACE = "2097152"
> 
> bitbake core-image-minimal
> bitbake core-image-minimal -c testimage
> 
> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> ---
>  meta/lib/oeqa/runtime/cases/ltp.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

Thanks for looking at this, it is interesting it now has the
parallelism support. I'll probably hold off this initially just so that
we can transition successfully but it then makes sense to tweak things.

I checked yocto-autobuilder-helper's config.json for the config we use
on the autobuilder and it is:

"IMAGE_INSTALL:append = ' ltp'",
"TEST_SUITES = 'ping ssh ltp ltp_compliance'",
"IMAGE_OVERHEAD_FACTOR = '1.0'",
"IMAGE_ROOTFS_EXTRA_SPACE = '1324288'",
"QB_MEM:forcevariable = '-m 1024'"

so we're running with a lot less memory than you configured but you
also have a lot more threads!

> diff --git a/meta/lib/oeqa/runtime/cases/ltp.py b/meta/lib/oeqa/runtime/cases/ltp.py
> index b855e76907..ae6a275110 100644
> --- a/meta/lib/oeqa/runtime/cases/ltp.py
> +++ b/meta/lib/oeqa/runtime/cases/ltp.py
> @@ -68,7 +68,9 @@ class LtpTest(LtpTestBase):
>              # LTP appends to log files, so ensure we start with a clean log
>              self.target.deleteFiles("/opt/ltp/results/", "%s.json" % ltp_group)
>  
> -            cmd = 'kirk --run-suite %s --json-report /opt/ltp/results/%s.json -n -d /opt/ltp --exec-timeout 20m' % (ltp_group, ltp_group)
> +            # LTP_WORKERS: number of parallel test workers (default: 1)
> +            workers = self.td.get('LTP_WORKERS', '1')
> +            cmd = 'kirk --run-suite %s --json-report /opt/ltp/results/%s.json -n -d /opt/ltp --exec-timeout 20m --workers %s' % (ltp_group, ltp_group, workers)

We probably need to namespace the variable a little more, such as 
OEQA_LTP_WORKERS so that people have a hint on where it is used.

Cheers,

Richard
Daniel Turull March 27, 2026, 3:41 p.m. UTC | #2 
Sure, we can keep this for later after the release.
I can have another one with the ltp_groups to be configurable and not hard code in the ltp.py

I'll send a new series for the patches that do not make if for this release.

Daniel

> -----Original Message-----
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
> Sent: Friday, 27 March 2026 16:37
> To: Daniel Turull <daniel.turull@ericsson.com>; openembedded-
> core@lists.openembedded.org
> Cc: bruce.ashfield@gmail.com; alex.kanavin@gmail.com;
> pratik.farkase@est.tech
> Subject: Re: [PATCH v2 6/6] oeqa: ltp - add LTP_WORKERS to allow parallelization
> 
> On Fri, 2026-03-27 at 13:27 +0100, daniel.turull@ericsson.com wrote:
> > From: Daniel Turull <daniel.turull@ericsson.com>
> >
> > Add new configurable variable to execute multiple workers in parallel.
> > kirk adds the option that was not available in runltp.
> >
> > Tested in qemux86-64 with configuration:
> >
> > IMAGE_CLASSES += "testimage"
> > CORE_IMAGE_EXTRA_INSTALL += "ltp openssh"
> > TEST_SUITES = "ping ssh ltp"
> > QB_MEM = "32768"
> > QB_CPU_KVM = "-cpu host -smp 16"
> > QB_SMP = "-smp 16"
> > LTP_WORKERS = "16"
> > IMAGE_ROOTFS_EXTRA_SPACE = "2097152"
> >
> > bitbake core-image-minimal
> > bitbake core-image-minimal -c testimage
> >
> > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> > ---
> >  meta/lib/oeqa/runtime/cases/ltp.py | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> Thanks for looking at this, it is interesting it now has the parallelism support. I'll
> probably hold off this initially just so that we can transition successfully but it then
> makes sense to tweak things.
> 
> I checked yocto-autobuilder-helper's config.json for the config we use on the
> autobuilder and it is:
> 
> "IMAGE_INSTALL:append = ' ltp'",
> "TEST_SUITES = 'ping ssh ltp ltp_compliance'", "IMAGE_OVERHEAD_FACTOR =
> '1.0'", "IMAGE_ROOTFS_EXTRA_SPACE = '1324288'", "QB_MEM:forcevariable = '-
> m 1024'"
> 
> so we're running with a lot less memory than you configured but you also have a
> lot more threads!
> 
> > diff --git a/meta/lib/oeqa/runtime/cases/ltp.py
> > b/meta/lib/oeqa/runtime/cases/ltp.py
> > index b855e76907..ae6a275110 100644
> > --- a/meta/lib/oeqa/runtime/cases/ltp.py
> > +++ b/meta/lib/oeqa/runtime/cases/ltp.py
> > @@ -68,7 +68,9 @@ class LtpTest(LtpTestBase):
> >              # LTP appends to log files, so ensure we start with a
> > clean log
> >              self.target.deleteFiles("/opt/ltp/results/", "%s.json" %
> > ltp_group)
> >
> > -            cmd = 'kirk --run-suite %s --json-report
> > /opt/ltp/results/%s.json -n -d /opt/ltp --exec-timeout 20m' %
> > (ltp_group, ltp_group)
> > +            # LTP_WORKERS: number of parallel test workers (default:
> > +1)
> > +            workers = self.td.get('LTP_WORKERS', '1')
> > +            cmd = 'kirk --run-suite %s --json-report
> > +/opt/ltp/results/%s.json -n -d /opt/ltp --exec-timeout 20m --workers
> > +%s' % (ltp_group, ltp_group, workers)
> 
> We probably need to namespace the variable a little more, such as
> OEQA_LTP_WORKERS so that people have a hint on where it is used.
> 
> Cheers,
> 
> Richard
diff mbox series

Patch

diff --git a/meta/lib/oeqa/runtime/cases/ltp.py b/meta/lib/oeqa/runtime/cases/ltp.py
index b855e76907..ae6a275110 100644
--- a/meta/lib/oeqa/runtime/cases/ltp.py
+++ b/meta/lib/oeqa/runtime/cases/ltp.py
@@ -68,7 +68,9 @@  class LtpTest(LtpTestBase):
             # LTP appends to log files, so ensure we start with a clean log
             self.target.deleteFiles("/opt/ltp/results/", "%s.json" % ltp_group)
 
-            cmd = 'kirk --run-suite %s --json-report /opt/ltp/results/%s.json -n -d /opt/ltp --exec-timeout 20m' % (ltp_group, ltp_group)
+            # LTP_WORKERS: number of parallel test workers (default: 1)
+            workers = self.td.get('LTP_WORKERS', '1')
+            cmd = 'kirk --run-suite %s --json-report /opt/ltp/results/%s.json -n -d /opt/ltp --exec-timeout 20m --workers %s' % (ltp_group, ltp_group, workers)
 
             starttime = time.time()
             (status, output) = self.target.run(cmd, timeout=1200)