[1/1] strace: 6.17 -> 6.18

From: Robert Yang <liezhi.yang@windriver.com>

The 6.17 has 27 failed test cases, and 6.18 only has 2 failed ones:
FAIL: bpf.gen.test
FAIL: bpf-v.gen.test

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/strace/{strace_6.17.bb => strace_6.18.bb} | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 rename meta/recipes-devtools/strace/{strace_6.17.bb => strace_6.18.bb} (96%)

On Tue, 16 Dec 2025 at 10:57, Robert Yang via lists.openembedded.org
<liezhi.yang=windriver.com@lists.openembedded.org> wrote:
> The 6.17 has 27 failed test cases, and 6.18 only has 2 failed ones:
> FAIL: bpf.gen.test
> FAIL: bpf-v.gen.test

This needs a bit more context. If the old version had failures, and
the new version still has failures, why aren't they seen in testing?
Where and how do they happen? Should they be fixed rather?

Alex

Hi Alexander,

On 12/16/25 19:29, Alexander Kanavin wrote:
> On Tue, 16 Dec 2025 at 10:57, Robert Yang via lists.openembedded.org
> <liezhi.yang=windriver.com@lists.openembedded.org> wrote:
>> The 6.17 has 27 failed test cases, and 6.18 only has 2 failed ones:
>> FAIL: bpf.gen.test
>> FAIL: bpf-v.gen.test
> 
> This needs a bit more context. If the old version had failures, and
> the new version still has failures, why aren't they seen in testing?

They are ptest failures, and there is no specific test for it, so we don't see
the failures. I did look into the failures, but didn't know how to fix that, the
newer version has less failures than the older one, so I sent the upgrade patch.

// Robert

> Where and how do they happen? Should they be fixed rather?
 > > Alex

On Wed, 17 Dec 2025 at 03:58, Robert Yang <liezhi.yang@windriver.com> wrote:
> >> The 6.17 has 27 failed test cases, and 6.18 only has 2 failed ones:
> >> FAIL: bpf.gen.test
> >> FAIL: bpf-v.gen.test
> >
> > This needs a bit more context. If the old version had failures, and
> > the new version still has failures, why aren't they seen in testing?
>
> They are ptest failures, and there is no specific test for it, so we don't see
> the failures. I did look into the failures, but didn't know how to fix that, the
> newer version has less failures than the older one, so I sent the upgrade patch.

But we do run strace ptests, and they do not fail, e.g. with latest
nightly master:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2761/steps/14/logs/stdio

Does bitbake -c testimage core-image-ptest-strace fail for you? How
did you set up and ran the tests to see them fail?

Alex

Hi Alexander,

On 12/17/25 16:41, Alexander Kanavin wrote:
> On Wed, 17 Dec 2025 at 03:58, Robert Yang <liezhi.yang@windriver.com> wrote:
>>>> The 6.17 has 27 failed test cases, and 6.18 only has 2 failed ones:
>>>> FAIL: bpf.gen.test
>>>> FAIL: bpf-v.gen.test
>>>
>>> This needs a bit more context. If the old version had failures, and
>>> the new version still has failures, why aren't they seen in testing?
>>
>> They are ptest failures, and there is no specific test for it, so we don't see
>> the failures. I did look into the failures, but didn't know how to fix that, the
>> newer version has less failures than the older one, so I sent the upgrade patch.
> 
> But we do run strace ptests, and they do not fail, e.g. with latest
> nightly master:
> 
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2761/steps/14/logs/stdio
> 
> Does bitbake -c testimage core-image-ptest-strace fail for you? How
> did you set up and ran the tests to see them fail?

You didn't see the failure is because core-image-ptest has PTEST_EXPECT_FAILURE
as reported by Qi:

https://lists.openembedded.org/g/openembedded-core/message/227411

I can see the failure after applied Qi's patches. Or we can manually see the
failures via:
# pest-runner strace

// Robert

> 
> Alex

On Tue, 23 Dec 2025 at 08:36, Robert Yang <liezhi.yang@windriver.com> wrote:

> > But we do run strace ptests, and they do not fail, e.g. with latest
> > nightly master:
> >
> > https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2761/steps/14/logs/stdio
> >
> > Does bitbake -c testimage core-image-ptest-strace fail for you? How
> > did you set up and ran the tests to see them fail?
>
> You didn't see the failure is because core-image-ptest has PTEST_EXPECT_FAILURE
> as reported by Qi:
>
> https://lists.openembedded.org/g/openembedded-core/message/227411
>
> I can see the failure after applied Qi's patches. Or we can manually see the
> failures via:
> # pest-runner strace

Even without Qi's patches, if a ptest failure happens, autobuilder
will report it as a warning, and for 6.17 those were not happening
either in autobuilder or for me locally. But 6.18 did indeed start to
fail in the two bpf tests, so we had to patch them out:

https://git.openembedded.org/openembedded-core/log/?h=master-next

So there's probably something in your configuration that triggers
those fails in 6.17 that doesn't happen in standard poky.

Alex

On 12/23/25 18:21, Alexander Kanavin wrote:
> On Tue, 23 Dec 2025 at 08:36, Robert Yang <liezhi.yang@windriver.com> wrote:
> 
>>> But we do run strace ptests, and they do not fail, e.g. with latest
>>> nightly master:
>>>
>>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2761/steps/14/logs/stdio
>>>
>>> Does bitbake -c testimage core-image-ptest-strace fail for you? How
>>> did you set up and ran the tests to see them fail?
>>
>> You didn't see the failure is because core-image-ptest has PTEST_EXPECT_FAILURE
>> as reported by Qi:
>>
>> https://lists.openembedded.org/g/openembedded-core/message/227411
>>
>> I can see the failure after applied Qi's patches. Or we can manually see the
>> failures via:
>> # pest-runner strace
> 
> Even without Qi's patches, if a ptest failure happens, autobuilder
> will report it as a warning, and for 6.17 those were not happening
> either in autobuilder or for me locally. But 6.18 did indeed start to
> fail in the two bpf tests, so we had to patch them out:
> 
> https://git.openembedded.org/openembedded-core/log/?h=master-next
> 
> So there's probably something in your configuration that triggers
> those fails in 6.17 that doesn't happen in standard poky.

You're right! All test cases are passed in 6.17:

# TOTAL: 1450
# PASS:  1121
# SKIP:  329
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0

So we should hold on this upgrading.

I didn't use core-image-ptest-strace in previous testing, but test it manually
with core-image-sato, so I got the failures on 6.17.


// Robert

> 
> Alex

On Wed, 24 Dec 2025 at 13:39, Robert Yang <liezhi.yang@windriver.com> wrote:
> You're right! All test cases are passed in 6.17:
>
> # TOTAL: 1450
> # PASS:  1121
> # SKIP:  329
> # XFAIL: 0
> # FAIL:  0
> # XPASS: 0
> # ERROR: 0
>
> So we should hold on this upgrading.
>
> I didn't use core-image-ptest-strace in previous testing, but test it manually
> with core-image-sato, so I got the failures on 6.17.

The upgrade has already merged, as it was needed for the kernel
updates. The two failing bpf tests were patched out so that reset can
pass.

Alex

On 12/24/25 21:58, Alexander Kanavin wrote:
> On Wed, 24 Dec 2025 at 13:39, Robert Yang <liezhi.yang@windriver.com> wrote:
>> You're right! All test cases are passed in 6.17:
>>
>> # TOTAL: 1450
>> # PASS:  1121
>> # SKIP:  329
>> # XFAIL: 0
>> # FAIL:  0
>> # XPASS: 0
>> # ERROR: 0
>>
>> So we should hold on this upgrading.
>>
>> I didn't use core-image-ptest-strace in previous testing, but test it manually
>> with core-image-sato, so I got the failures on 6.17.
> 
> The upgrade has already merged, as it was needed for the kernel
> updates. The two failing bpf tests were patched out so that reset can
> pass.

Thank you very much!

// Robert

> 
> Alex