| Message ID | 20250802131227.677594-2-skandigraun@gmail.com |
|---|---|
| State | Accepted, archived |
| Commit | 02ed7fad85463840c46b6c0fa0ac9decef77c503 |
| Headers | show |
| Series | [1/2] dpkg: upgrade 1.22.11 -> 1.22.21 | expand |
On Sat Aug 2, 2025 at 3:12 PM CEST, Gyorgy Sarvari via lists.openembedded.org wrote: > Added it to slow tests, as it takes betwen 165 and 190 seconds on my > machine to execute (qemux86-64 + kvm). > > The test folder's Makefile contains a list of passing, failing and manual > tests. By default, only the expected-to-pass tests are executed by the > Makefile (unless magic environment variable is set). > > The run-ptest script mimics the default behavior of executing the > expected-to-pass tests, however they are executed one by one, instead of > running them as one batch - that way it is easier to determine exactly > which tests pass and which fail. > > One other thing that might worth a note, is that the tests folder that needs to be > installed contains a number of subfolders called "DEBIAN". When packaging them > at least with rpm, these folders are omitted from the package. > However these are essential for the tests, as they contain test data. As a > workaround, these folders are renamed during installation to DEBIAN-ptest, > and before execution the run-ptest script restores their names. > > Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> > --- Hi Gyorgy, Thanks for your patch. It looks like some of these ptests are failing: WARNING: core-image-ptest-dpkg-1.0-r0 do_testimage: There were failing ptests. ... AssertionError: ptests which had no test results: ['dpkg'] Cannot run ptests without @expectedFailure as ptests are expected to fail https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/3164697/raw_inline https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/3164752/raw_inline https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/3164788/raw_inline Ptest results can be found here: https://valkyrie.yocto.io/pub/non-release/20250805-32/testresults/qemuarm64-ptest/core-image-ptest-dpkg/ https://valkyrie.yocto.io/pub/non-release/20250805-32/testresults/qemuriscv64-ptest/core-image-ptest-dpkg/ https://valkyrie.yocto.io/pub/non-release/20250805-32/testresults/qemux86-64-ptest/core-image-ptest-dpkg/ Best regards, Mathieu
On 8/5/25 10:14, Mathieu Dubois-Briand wrote: > On Sat Aug 2, 2025 at 3:12 PM CEST, Gyorgy Sarvari via lists.openembedded.org wrote: >> Added it to slow tests, as it takes betwen 165 and 190 seconds on my >> machine to execute (qemux86-64 + kvm). >> >> The test folder's Makefile contains a list of passing, failing and manual >> tests. By default, only the expected-to-pass tests are executed by the >> Makefile (unless magic environment variable is set). >> >> The run-ptest script mimics the default behavior of executing the >> expected-to-pass tests, however they are executed one by one, instead of >> running them as one batch - that way it is easier to determine exactly >> which tests pass and which fail. >> >> One other thing that might worth a note, is that the tests folder that needs to be >> installed contains a number of subfolders called "DEBIAN". When packaging them >> at least with rpm, these folders are omitted from the package. >> However these are essential for the tests, as they contain test data. As a >> workaround, these folders are renamed during installation to DEBIAN-ptest, >> and before execution the run-ptest script restores their names. >> >> Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> >> --- > Hi Gyorgy, > > Thanks for your patch. > > It looks like some of these ptests are failing: > > WARNING: core-image-ptest-dpkg-1.0-r0 do_testimage: There were failing ptests. > ... > AssertionError: > ptests which had no test results: > ['dpkg'] > > Cannot run ptests without @expectedFailure as ptests are expected to fail > > https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/3164697/raw_inline > https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/3164752/raw_inline > https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/3164788/raw_inline > > Ptest results can be found here: > > https://valkyrie.yocto.io/pub/non-release/20250805-32/testresults/qemuarm64-ptest/core-image-ptest-dpkg/ > https://valkyrie.yocto.io/pub/non-release/20250805-32/testresults/qemuriscv64-ptest/core-image-ptest-dpkg/ > https://valkyrie.yocto.io/pub/non-release/20250805-32/testresults/qemux86-64-ptest/core-image-ptest-dpkg/ > > Best regards, > Mathieu > I think I see the problem, the output format of the run-ptest seems to be incorrect. I'm afraid I made the same mistake with librsvg ptests too. Thanks for the heads up, will send a v2 for both dpkg and librsvg ptest patches.
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc index 4451e00b48..7e08225b99 100644 --- a/meta/conf/distro/include/ptest-packagelists.inc +++ b/meta/conf/distro/include/ptest-packagelists.inc @@ -100,6 +100,7 @@ PTESTS_SLOW = "\ coreutils \ curl \ dbus \ + dpkg \ e2fsprogs \ elfutils \ gettext \ diff --git a/meta/recipes-devtools/dpkg/dpkg.inc b/meta/recipes-devtools/dpkg/dpkg.inc index dea1894a4c..48dc069740 100644 --- a/meta/recipes-devtools/dpkg/dpkg.inc +++ b/meta/recipes-devtools/dpkg/dpkg.inc @@ -11,7 +11,7 @@ RDEPENDS:${PN}:class-native = "" UPSTREAM_CHECK_GITTAGREGEX = "(?P<pver>(\d+(\.\d+)+))" -inherit autotools gettext perlnative pkgconfig perl-version update-alternatives bash-completion +inherit autotools gettext perlnative pkgconfig perl-version update-alternatives bash-completion ptest PERL:class-native = "${STAGING_BINDIR_NATIVE}/perl-native/perl" @@ -56,6 +56,19 @@ do_install:append () { fi } +do_install_ptest () { + install -d ${D}${PTEST_PATH}/tests ${D}${PTEST_PATH}/src/sh + install -m 0755 ${S}/src/sh/dpkg-error.sh ${D}${PTEST_PATH}/src/sh/ + cp --preserve=mode,timestamps -R ${S}/tests/* ${D}${PTEST_PATH}/tests + + # The folder has a number of test folders called DEBIAN, but these are + # explicitly omitted from some packages, e.g. from rpm. + # However these are essential for the tests, so rename them to DEBIAN-ptest, and at runtime + # from the run-ptest script restore their names + find ${D}${PTEST_PATH}/tests -name DEBIAN -type d | xargs -n1 -I{} mv "{}" "{}-ptest" +} + + PROV = "virtual/update-alternatives" PROV:class-native = "" PROV:class-nativesdk = "" @@ -90,6 +103,8 @@ RDEPENDS:${PN}-perl += "perl-module-carp perl-module-constant \ perl-module-time-hires perl-module-time-piece \ perl-module-xsloader" +RDEPENDS:${PN}-ptest += "make coreutils findutils" + # Split out start-stop-daemon to its own package. Note that it # is installed in a different directory than the one used for # the bitbake version. diff --git a/meta/recipes-devtools/dpkg/dpkg/run-ptest b/meta/recipes-devtools/dpkg/dpkg/run-ptest new file mode 100644 index 0000000000..6a4f16a6e1 --- /dev/null +++ b/meta/recipes-devtools/dpkg/dpkg/run-ptest @@ -0,0 +1,37 @@ +#!/bin/sh + +rm -f dpkg-test.log +cd tests + +RESULT=0 + +# During installation the "DEBIAN" folders are renamed to avoid +# problems with packaging. Restore their names here. +for d in $(find . -name DEBIAN-ptest -type d); do + dname=$(dirname "$d") + mv ${d} ${dname}/DEBIAN +done + +# This allows running tests that require root access (which we have) +export DPKG_AS_ROOT=1 + +# The test suite has also a number of test that are expected to fail, +# and some manual tests also. By default, only the tests are executed +# by the Makefile which are expected to pass - do the same also. +# +# It is possible to run all tests with "make test" also, however +# it is hard to parse the actual results, what is broken, what is not +# due to the amount of output. + +for test in $(grep "TESTS_PASS +=" Makefile | cut -f3 -d" "); do + make ${test}-test >> ../dpkg-test.log 2>&1 + + if [ $? -eq 0 ]; then + echo OK: $test + else + echo FAIL: $test + RESULT=1 + fi +done + +exit ${RESULT} diff --git a/meta/recipes-devtools/dpkg/dpkg_1.22.21.bb b/meta/recipes-devtools/dpkg/dpkg_1.22.21.bb index cc30b71109..d793c26d57 100644 --- a/meta/recipes-devtools/dpkg/dpkg_1.22.21.bb +++ b/meta/recipes-devtools/dpkg/dpkg_1.22.21.bb @@ -6,6 +6,7 @@ SRC_URI = "git://salsa.debian.org/dpkg-team/dpkg.git;protocol=https;branch=1.22. file://remove-tar-no-timestamp.patch \ file://arch_pm.patch \ file://add_armeb_triplet_entry.patch \ + file://run-ptest \ file://0002-Adapt-to-linux-wrs-kernel-version-which-has-characte.patch \ file://0001-script.c-avoid-use-of-chroot.patch \ file://0004-The-lutimes-function-doesn-t-work-properly-for-all-s.patch \
Added it to slow tests, as it takes betwen 165 and 190 seconds on my machine to execute (qemux86-64 + kvm). The test folder's Makefile contains a list of passing, failing and manual tests. By default, only the expected-to-pass tests are executed by the Makefile (unless magic environment variable is set). The run-ptest script mimics the default behavior of executing the expected-to-pass tests, however they are executed one by one, instead of running them as one batch - that way it is easier to determine exactly which tests pass and which fail. One other thing that might worth a note, is that the tests folder that needs to be installed contains a number of subfolders called "DEBIAN". When packaging them at least with rpm, these folders are omitted from the package. However these are essential for the tests, as they contain test data. As a workaround, these folders are renamed during installation to DEBIAN-ptest, and before execution the run-ptest script restores their names. Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> --- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-devtools/dpkg/dpkg.inc | 17 ++++++++- meta/recipes-devtools/dpkg/dpkg/run-ptest | 37 +++++++++++++++++++ meta/recipes-devtools/dpkg/dpkg_1.22.21.bb | 1 + 4 files changed, 55 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-devtools/dpkg/dpkg/run-ptest