| Message ID | 20240126054820.1656852-1-chi.xu@windriver.com |
|---|---|
| State | Accepted, archived |
| Commit | 2704983f972e4fe1d4e0bee8491a07eb4f629346 |
| Headers | show |
| Series | [v3] xz: Add ptest support | expand |
Hello, I've had a reproducibility failure: https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20240203-fdpg888c/packages/diff-html/ On 26/01/2024 13:48:20+0800, Chi Xu wrote: > There are two types of cases: executables and POSIX shell scripts. > All test cases PASS. > Add xz-ptest to PTESTS_FAST because test duration less than 30s > on qemux86-64. > > root@qemux86-64:~# ptest-runner xz > START: ptest-runner > 2024-01-26T03:32 > BEGIN: /usr/lib/xz/ptest > === test_bcj_exact_size.c === > PASS: test_exact_size > PASS: test_empty_block > --- > === END === > ... > === test_vli.c === > PASS: test_lzma_vli_size > PASS: test_lzma_vli_encode > PASS: test_lzma_vli_decode > --- > === END === > PASS: test_files.sh > PASS: test_scripts.sh > PASS: test_compress.sh > DURATION: 18 > END: /usr/lib/xz/ptest > 2024-01-26T03:33 > STOP: ptest-runner > TOTAL: 1 FAIL: 0 > > Signed-off-by: Xu, Chi <chi.xu@windriver.com> > --- > .../distro/include/ptest-packagelists.inc | 1 + > meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ > meta/recipes-extended/xz/xz_5.4.5.bb | 28 +++++++++++++++++-- > 3 files changed, 53 insertions(+), 2 deletions(-) > create mode 100644 meta/recipes-extended/xz/xz/run-ptest > > diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc > index a1643c5548..68a3f63a6a 100644 > --- a/meta/conf/distro/include/ptest-packagelists.inc > +++ b/meta/conf/distro/include/ptest-packagelists.inc > @@ -77,6 +77,7 @@ PTESTS_FAST = "\ > sed \ > slang \ > wayland \ > + xz \ > zlib \ > libexif \ > " > diff --git a/meta/recipes-extended/xz/xz/run-ptest b/meta/recipes-extended/xz/xz/run-ptest > new file mode 100644 > index 0000000000..cc8ba92016 > --- /dev/null > +++ b/meta/recipes-extended/xz/xz/run-ptest > @@ -0,0 +1,26 @@ > +#!/bin/bash > + > +ptestdir=$(dirname "$(readlink -f "$0")") > +cd "$ptestdir"/tests || exit > + > +# executables test cases > +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') > +for i in ${tests_exec}; do > + ./$i > +done > + > +# POSIX shell script test cases > +export srcdir=${ptestdir}/tests > +test_shell="test_files.sh test_scripts.sh test_compress.sh" > +for j in ${test_shell}; do > + if [ $j == "test_compress.sh" ]; then > + ./$j create_compress_files > + else > + ./$j > + fi > + if [ $? -eq 0 ]; then > + echo "PASS: $j" > + else > + echo "FAIL: $j" > + fi > +done > diff --git a/meta/recipes-extended/xz/xz_5.4.5.bb b/meta/recipes-extended/xz/xz_5.4.5.bb > index 09eade8be0..0f0849c3e5 100644 > --- a/meta/recipes-extended/xz/xz_5.4.5.bb > +++ b/meta/recipes-extended/xz/xz_5.4.5.bb > @@ -24,13 +24,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \ > file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \ > " > > -SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" > +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ > + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest', '', d)} \ > + " > SRC_URI[sha256sum] = "135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d6" > UPSTREAM_CHECK_REGEX = "xz-(?P<pver>\d+(\.\d+)+)\.tar" > > CACHED_CONFIGUREVARS += "gl_cv_posix_shell=/bin/sh" > > -inherit autotools gettext > +inherit autotools gettext ptest > > PACKAGES =+ "liblzma" > > @@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ > lzma lzcat unlzma" > > BBCLASSEXTEND = "native nativesdk" > + > +RDEPENDS:${PN}-ptest += "bash file" > + > +do_compile_ptest() { > + oe_runmake check TESTS= > +} > + > +do_install_ptest () { > + install -d ${D}${PTEST_PATH}/tests > + find ${B}/tests/.libs -type f -executable -exec cp {} ${D}${PTEST_PATH}/tests \; > + cp ${B}/config.h ${D}${PTEST_PATH} > + for i in files xzgrep_expected_output test_files.sh test_scripts.sh test_compress.sh; do > + cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests > + done > + mkdir -p ${D}${PTEST_PATH}/src/xz > + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz > + mkdir -p ${D}${PTEST_PATH}/src/xzdec > + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec > + mkdir -p ${D}${PTEST_PATH}/src/scripts > + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff > + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep > +} > -- > 2.25.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#194345): https://lists.openembedded.org/g/openembedded-core/message/194345 > Mute This Topic: https://lists.openembedded.org/mt/103971280/3617179 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Hello, I'm checking this issue, but can not reproduce it on my host now. I tried to build and rebuild xz-ptest on ubuntu 20.04 and ubuntu 22.04, xz-ptest-5.4.5-r0.core2_64.rpm are same. Do you have any suggestions? Ubuntu 20.04 root@localhost:/root$ git clone git://git.yoctoproject.org/poky root@localhost:/root/poky$ cd poky root@localhost:/root/poky$ source oe-init-build-env root@localhost:/root/poky/build$ bitbake xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:27 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ bitbake -c cleansstate xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:32 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm Ubuntu 22.04 root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:56 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ bitbake -c cleansstate xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:59 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm Thanks, Chi > -----Original Message----- > From: Alexandre Belloni <alexandre.belloni@bootlin.com> > Sent: Sunday, February 4, 2024 5:23 AM > To: Xu, Chi <Chi.Xu@windriver.com> > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH v3] xz: Add ptest support > > CAUTION: This email comes from a non Wind River email account! > Do not click links or open attachments unless you recognize the sender and > know the content is safe. > > Hello, > > I've had a reproducibility failure: > > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20240203- > fdpg888c/packages/diff-html/ > > On 26/01/2024 13:48:20+0800, Chi Xu wrote: > > There are two types of cases: executables and POSIX shell scripts. > > All test cases PASS. > > Add xz-ptest to PTESTS_FAST because test duration less than 30s on > > qemux86-64. > > > > root@qemux86-64:~# ptest-runner xz > > START: ptest-runner > > 2024-01-26T03:32 > > BEGIN: /usr/lib/xz/ptest > > === test_bcj_exact_size.c === > > PASS: test_exact_size > > PASS: test_empty_block > > --- > > === END === > > ... > > === test_vli.c === > > PASS: test_lzma_vli_size > > PASS: test_lzma_vli_encode > > PASS: test_lzma_vli_decode > > --- > > === END === > > PASS: test_files.sh > > PASS: test_scripts.sh > > PASS: test_compress.sh > > DURATION: 18 > > END: /usr/lib/xz/ptest > > 2024-01-26T03:33 > > STOP: ptest-runner > > TOTAL: 1 FAIL: 0 > > > > Signed-off-by: Xu, Chi <chi.xu@windriver.com> > > --- > > .../distro/include/ptest-packagelists.inc | 1 + > > meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ > > meta/recipes-extended/xz/xz_5.4.5.bb | 28 +++++++++++++++++-- > > 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 > > meta/recipes-extended/xz/xz/run-ptest > > > > diff --git a/meta/conf/distro/include/ptest-packagelists.inc > > b/meta/conf/distro/include/ptest-packagelists.inc > > index a1643c5548..68a3f63a6a 100644 > > --- a/meta/conf/distro/include/ptest-packagelists.inc > > +++ b/meta/conf/distro/include/ptest-packagelists.inc > > @@ -77,6 +77,7 @@ PTESTS_FAST = "\ > > sed \ > > slang \ > > wayland \ > > + xz \ > > zlib \ > > libexif \ > > " > > diff --git a/meta/recipes-extended/xz/xz/run-ptest > > b/meta/recipes-extended/xz/xz/run-ptest > > new file mode 100644 > > index 0000000000..cc8ba92016 > > --- /dev/null > > +++ b/meta/recipes-extended/xz/xz/run-ptest > > @@ -0,0 +1,26 @@ > > +#!/bin/bash > > + > > +ptestdir=$(dirname "$(readlink -f "$0")") cd "$ptestdir"/tests || > > +exit > > + > > +# executables test cases > > +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') > > +for i in ${tests_exec}; do > > + ./$i > > +done > > + > > +# POSIX shell script test cases > > +export srcdir=${ptestdir}/tests > > +test_shell="test_files.sh test_scripts.sh test_compress.sh" > > +for j in ${test_shell}; do > > + if [ $j == "test_compress.sh" ]; then > > + ./$j create_compress_files > > + else > > + ./$j > > + fi > > + if [ $? -eq 0 ]; then > > + echo "PASS: $j" > > + else > > + echo "FAIL: $j" > > + fi > > +done > > diff --git a/meta/recipes-extended/xz/xz_5.4.5.bb > > b/meta/recipes-extended/xz/xz_5.4.5.bb > > index 09eade8be0..0f0849c3e5 100644 > > --- a/meta/recipes-extended/xz/xz_5.4.5.bb > > +++ b/meta/recipes-extended/xz/xz_5.4.5.bb > > @@ -24,13 +24,15 @@ LIC_FILES_CHKSUM = > "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \ > > > file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \ > > " > > > > -SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" > > +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ > > + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest', '', d)} \ > > + " > > SRC_URI[sha256sum] = > "135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d > 6" > > UPSTREAM_CHECK_REGEX = "xz-(?P<pver>\d+(\.\d+)+)\.tar" > > > > CACHED_CONFIGUREVARS += "gl_cv_posix_shell=/bin/sh" > > > > -inherit autotools gettext > > +inherit autotools gettext ptest > > > > PACKAGES =+ "liblzma" > > > > @@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ > > lzma lzcat unlzma" > > > > BBCLASSEXTEND = "native nativesdk" > > + > > +RDEPENDS:${PN}-ptest += "bash file" > > + > > +do_compile_ptest() { > > + oe_runmake check TESTS= > > +} > > + > > +do_install_ptest () { > > + install -d ${D}${PTEST_PATH}/tests > > + find ${B}/tests/.libs -type f -executable -exec cp {} > ${D}${PTEST_PATH}/tests \; > > + cp ${B}/config.h ${D}${PTEST_PATH} > > + for i in files xzgrep_expected_output test_files.sh test_scripts.sh > test_compress.sh; do > > + cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests > > + done > > + mkdir -p ${D}${PTEST_PATH}/src/xz > > + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz > > + mkdir -p ${D}${PTEST_PATH}/src/xzdec > > + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec > > + mkdir -p ${D}${PTEST_PATH}/src/scripts > > + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff > > + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep > > +} > > -- > > 2.25.1 > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#194345): > > https://lists.openembedded.org/g/openembedded-core/message/194345 > > Mute This Topic: https://lists.openembedded.org/mt/103971280/3617179 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded- > core/unsub > > [alexandre.belloni@bootlin.com] > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > -- > Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel > engineering https://bootlin.com
Hi, FYI, since the reproducibility problem only impact the RPM package, it might be related to the problem noticed by Richard last night on IRC: [11:35:32 pm] <RP> kanavin: It looks like it runs file on the files within the archive but for some reason it isn't sorting that in the header, so probably an unsorted directory listing somewhere [11:58:34 pm] <RP> kanavin: looking at package_rpm and it's os.walk, there is no sorting in there at all. I suspect that leads to non-determinstic ordering of the files sections in the spec files and I suspect that in turn leads rpm to reorder headers (https://libera.irclog.whitequark.org/yocto/2024-02-04#35724802;) Le lun. 5 févr. 2024 à 07:10, Chi Xu <chi.xu@eng.windriver.com> a écrit : > Hello, > > I'm checking this issue, but can not reproduce it on my host now. > I tried to build and rebuild xz-ptest on ubuntu 20.04 and ubuntu 22.04, > xz-ptest-5.4.5-r0.core2_64.rpm are same. > Do you have any suggestions? > > Ubuntu 20.04 > root@localhost:/root$ git clone git://git.yoctoproject.org/poky > root@localhost:/root/poky$ cd poky > root@localhost:/root/poky$ source oe-init-build-env > root@localhost:/root/poky/build$ bitbake xz > root@localhost:/root/poky/build$ ls -l > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > -rw-r--r-- 2 root root 162489 Feb 5 13:27 > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > root@localhost:/root/poky/build$ md5sum > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > b26fd0014d1c23f6e3d9bf8704137b82 > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > root@localhost:/root/poky/build$ bitbake -c cleansstate xz > root@localhost:/root/poky/build$ ls -l > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > -rw-r--r-- 2 root root 162489 Feb 5 13:32 > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > root@localhost:/root/poky/build$ md5sum > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > b26fd0014d1c23f6e3d9bf8704137b82 > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > > Ubuntu 22.04 > root@localhost:/root/poky/build$ ls -l > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > -rw-r--r-- 2 root root 162489 Feb 5 13:56 > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > root@localhost:/root/poky/build$ md5sum > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > b26fd0014d1c23f6e3d9bf8704137b82 > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > root@localhost:/root/poky/build$ bitbake -c cleansstate xz > root@localhost:/root/poky/build$ ls -l > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > -rw-r--r-- 2 root root 162489 Feb 5 13:59 > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > root@localhost:/root/poky/build$ md5sum > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > b26fd0014d1c23f6e3d9bf8704137b82 > tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm > > Thanks, > Chi > > > -----Original Message----- > > From: Alexandre Belloni <alexandre.belloni@bootlin.com> > > Sent: Sunday, February 4, 2024 5:23 AM > > To: Xu, Chi <Chi.Xu@windriver.com> > > Cc: openembedded-core@lists.openembedded.org > > Subject: Re: [OE-core] [PATCH v3] xz: Add ptest support > > > > CAUTION: This email comes from a non Wind River email account! > > Do not click links or open attachments unless you recognize the sender > and > > know the content is safe. > > > > Hello, > > > > I've had a reproducibility failure: > > > > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20240203- > > fdpg888c/packages/diff-html/ > > > > On 26/01/2024 13:48:20+0800, Chi Xu wrote: > > > There are two types of cases: executables and POSIX shell scripts. > > > All test cases PASS. > > > Add xz-ptest to PTESTS_FAST because test duration less than 30s on > > > qemux86-64. > > > > > > root@qemux86-64:~# ptest-runner xz > > > START: ptest-runner > > > 2024-01-26T03:32 > > > BEGIN: /usr/lib/xz/ptest > > > === test_bcj_exact_size.c === > > > PASS: test_exact_size > > > PASS: test_empty_block > > > --- > > > === END === > > > ... > > > === test_vli.c === > > > PASS: test_lzma_vli_size > > > PASS: test_lzma_vli_encode > > > PASS: test_lzma_vli_decode > > > --- > > > === END === > > > PASS: test_files.sh > > > PASS: test_scripts.sh > > > PASS: test_compress.sh > > > DURATION: 18 > > > END: /usr/lib/xz/ptest > > > 2024-01-26T03:33 > > > STOP: ptest-runner > > > TOTAL: 1 FAIL: 0 > > > > > > Signed-off-by: Xu, Chi <chi.xu@windriver.com> > > > --- > > > .../distro/include/ptest-packagelists.inc | 1 + > > > meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ > > > meta/recipes-extended/xz/xz_5.4.5.bb | 28 > +++++++++++++++++-- > > > 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 > > > meta/recipes-extended/xz/xz/run-ptest > > > > > > diff --git a/meta/conf/distro/include/ptest-packagelists.inc > > > b/meta/conf/distro/include/ptest-packagelists.inc > > > index a1643c5548..68a3f63a6a 100644 > > > --- a/meta/conf/distro/include/ptest-packagelists.inc > > > +++ b/meta/conf/distro/include/ptest-packagelists.inc > > > @@ -77,6 +77,7 @@ PTESTS_FAST = "\ > > > sed \ > > > slang \ > > > wayland \ > > > + xz \ > > > zlib \ > > > libexif \ > > > " > > > diff --git a/meta/recipes-extended/xz/xz/run-ptest > > > b/meta/recipes-extended/xz/xz/run-ptest > > > new file mode 100644 > > > index 0000000000..cc8ba92016 > > > --- /dev/null > > > +++ b/meta/recipes-extended/xz/xz/run-ptest > > > @@ -0,0 +1,26 @@ > > > +#!/bin/bash > > > + > > > +ptestdir=$(dirname "$(readlink -f "$0")") cd "$ptestdir"/tests || > > > +exit > > > + > > > +# executables test cases > > > +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') > > > +for i in ${tests_exec}; do > > > + ./$i > > > +done > > > + > > > +# POSIX shell script test cases > > > +export srcdir=${ptestdir}/tests > > > +test_shell="test_files.sh test_scripts.sh test_compress.sh" > > > +for j in ${test_shell}; do > > > + if [ $j == "test_compress.sh" ]; then > > > + ./$j create_compress_files > > > + else > > > + ./$j > > > + fi > > > + if [ $? -eq 0 ]; then > > > + echo "PASS: $j" > > > + else > > > + echo "FAIL: $j" > > > + fi > > > +done > > > diff --git a/meta/recipes-extended/xz/xz_5.4.5.bb > > > b/meta/recipes-extended/xz/xz_5.4.5.bb > > > index 09eade8be0..0f0849c3e5 100644 > > > --- a/meta/recipes-extended/xz/xz_5.4.5.bb > > > +++ b/meta/recipes-extended/xz/xz_5.4.5.bb > > > @@ -24,13 +24,15 @@ LIC_FILES_CHKSUM = > > "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \ > > > > > file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \ > > > " > > > > > > -SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" > > > +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ > > > + ${@bb.utils.contains('PTEST_ENABLED', '1', > 'file://run-ptest', '', d)} \ > > > + " > > > SRC_URI[sha256sum] = > > "135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d > > 6" > > > UPSTREAM_CHECK_REGEX = "xz-(?P<pver>\d+(\.\d+)+)\.tar" > > > > > > CACHED_CONFIGUREVARS += "gl_cv_posix_shell=/bin/sh" > > > > > > -inherit autotools gettext > > > +inherit autotools gettext ptest > > > > > > PACKAGES =+ "liblzma" > > > > > > @@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ > > > lzma lzcat unlzma" > > > > > > BBCLASSEXTEND = "native nativesdk" > > > + > > > +RDEPENDS:${PN}-ptest += "bash file" > > > + > > > +do_compile_ptest() { > > > + oe_runmake check TESTS= > > > +} > > > + > > > +do_install_ptest () { > > > + install -d ${D}${PTEST_PATH}/tests > > > + find ${B}/tests/.libs -type f -executable -exec cp {} > > ${D}${PTEST_PATH}/tests \; > > > + cp ${B}/config.h ${D}${PTEST_PATH} > > > + for i in files xzgrep_expected_output test_files.sh > test_scripts.sh > > test_compress.sh; do > > > + cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests > > > + done > > > + mkdir -p ${D}${PTEST_PATH}/src/xz > > > + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz > > > + mkdir -p ${D}${PTEST_PATH}/src/xzdec > > > + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec > > > + mkdir -p ${D}${PTEST_PATH}/src/scripts > > > + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff > > > + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep > > > +} > > > -- > > > 2.25.1 > > > > > > > > > > > > > > > > > > > > -- > > Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel > > engineering https://bootlin.com > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#194916): > https://lists.openembedded.org/g/openembedded-core/message/194916 > Mute This Topic: https://lists.openembedded.org/mt/103971280/4316185 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > yoann.congal@smile.fr] > -=-=-=-=-=-=-=-=-=-=-=- > >
I just sent two patches for this. Alex On Mon, 5 Feb 2024 at 07:39, Yoann Congal <yoann.congal@smile.fr> wrote: > > Hi, > > FYI, since the reproducibility problem only impact the RPM package, it might be related to the problem noticed by Richard last night on IRC: > [11:35:32 pm] <RP> kanavin: It looks like it runs file on the files within the archive but for some reason it isn't sorting that in the header, so probably an unsorted directory listing somewhere > [11:58:34 pm] <RP> kanavin: looking at package_rpm and it's os.walk, there is no sorting in there at all. I suspect that leads to non-determinstic ordering of the files sections in the spec files and I suspect that in turn leads rpm to reorder headers > (https://libera.irclog.whitequark.org/yocto/2024-02-04#35724802;) > > Le lun. 5 févr. 2024 à 07:10, Chi Xu <chi.xu@eng.windriver.com> a écrit : >> >> Hello, >> >> I'm checking this issue, but can not reproduce it on my host now. >> I tried to build and rebuild xz-ptest on ubuntu 20.04 and ubuntu 22.04, >> xz-ptest-5.4.5-r0.core2_64.rpm are same. >> Do you have any suggestions? >> >> Ubuntu 20.04 >> root@localhost:/root$ git clone git://git.yoctoproject.org/poky >> root@localhost:/root/poky$ cd poky >> root@localhost:/root/poky$ source oe-init-build-env >> root@localhost:/root/poky/build$ bitbake xz >> root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> -rw-r--r-- 2 root root 162489 Feb 5 13:27 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> root@localhost:/root/poky/build$ bitbake -c cleansstate xz >> root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> -rw-r--r-- 2 root root 162489 Feb 5 13:32 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> >> Ubuntu 22.04 >> root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> -rw-r--r-- 2 root root 162489 Feb 5 13:56 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> root@localhost:/root/poky/build$ bitbake -c cleansstate xz >> root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> -rw-r--r-- 2 root root 162489 Feb 5 13:59 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm >> >> Thanks, >> Chi >> >> > -----Original Message----- >> > From: Alexandre Belloni <alexandre.belloni@bootlin.com> >> > Sent: Sunday, February 4, 2024 5:23 AM >> > To: Xu, Chi <Chi.Xu@windriver.com> >> > Cc: openembedded-core@lists.openembedded.org >> > Subject: Re: [OE-core] [PATCH v3] xz: Add ptest support >> > >> > CAUTION: This email comes from a non Wind River email account! >> > Do not click links or open attachments unless you recognize the sender and >> > know the content is safe. >> > >> > Hello, >> > >> > I've had a reproducibility failure: >> > >> > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20240203- >> > fdpg888c/packages/diff-html/ >> > >> > On 26/01/2024 13:48:20+0800, Chi Xu wrote: >> > > There are two types of cases: executables and POSIX shell scripts. >> > > All test cases PASS. >> > > Add xz-ptest to PTESTS_FAST because test duration less than 30s on >> > > qemux86-64. >> > > >> > > root@qemux86-64:~# ptest-runner xz >> > > START: ptest-runner >> > > 2024-01-26T03:32 >> > > BEGIN: /usr/lib/xz/ptest >> > > === test_bcj_exact_size.c === >> > > PASS: test_exact_size >> > > PASS: test_empty_block >> > > --- >> > > === END === >> > > ... >> > > === test_vli.c === >> > > PASS: test_lzma_vli_size >> > > PASS: test_lzma_vli_encode >> > > PASS: test_lzma_vli_decode >> > > --- >> > > === END === >> > > PASS: test_files.sh >> > > PASS: test_scripts.sh >> > > PASS: test_compress.sh >> > > DURATION: 18 >> > > END: /usr/lib/xz/ptest >> > > 2024-01-26T03:33 >> > > STOP: ptest-runner >> > > TOTAL: 1 FAIL: 0 >> > > >> > > Signed-off-by: Xu, Chi <chi.xu@windriver.com> >> > > --- >> > > .../distro/include/ptest-packagelists.inc | 1 + >> > > meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ >> > > meta/recipes-extended/xz/xz_5.4.5.bb | 28 +++++++++++++++++-- >> > > 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 >> > > meta/recipes-extended/xz/xz/run-ptest >> > > >> > > diff --git a/meta/conf/distro/include/ptest-packagelists.inc >> > > b/meta/conf/distro/include/ptest-packagelists.inc >> > > index a1643c5548..68a3f63a6a 100644 >> > > --- a/meta/conf/distro/include/ptest-packagelists.inc >> > > +++ b/meta/conf/distro/include/ptest-packagelists.inc >> > > @@ -77,6 +77,7 @@ PTESTS_FAST = "\ >> > > sed \ >> > > slang \ >> > > wayland \ >> > > + xz \ >> > > zlib \ >> > > libexif \ >> > > " >> > > diff --git a/meta/recipes-extended/xz/xz/run-ptest >> > > b/meta/recipes-extended/xz/xz/run-ptest >> > > new file mode 100644 >> > > index 0000000000..cc8ba92016 >> > > --- /dev/null >> > > +++ b/meta/recipes-extended/xz/xz/run-ptest >> > > @@ -0,0 +1,26 @@ >> > > +#!/bin/bash >> > > + >> > > +ptestdir=$(dirname "$(readlink -f "$0")") cd "$ptestdir"/tests || >> > > +exit >> > > + >> > > +# executables test cases >> > > +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') >> > > +for i in ${tests_exec}; do >> > > + ./$i >> > > +done >> > > + >> > > +# POSIX shell script test cases >> > > +export srcdir=${ptestdir}/tests >> > > +test_shell="test_files.sh test_scripts.sh test_compress.sh" >> > > +for j in ${test_shell}; do >> > > + if [ $j == "test_compress.sh" ]; then >> > > + ./$j create_compress_files >> > > + else >> > > + ./$j >> > > + fi >> > > + if [ $? -eq 0 ]; then >> > > + echo "PASS: $j" >> > > + else >> > > + echo "FAIL: $j" >> > > + fi >> > > +done >> > > diff --git a/meta/recipes-extended/xz/xz_5.4.5.bb >> > > b/meta/recipes-extended/xz/xz_5.4.5.bb >> > > index 09eade8be0..0f0849c3e5 100644 >> > > --- a/meta/recipes-extended/xz/xz_5.4.5.bb >> > > +++ b/meta/recipes-extended/xz/xz_5.4.5.bb >> > > @@ -24,13 +24,15 @@ LIC_FILES_CHKSUM = >> > "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \ >> > > >> > file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \ >> > > " >> > > >> > > -SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" >> > > +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ >> > > + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest', '', d)} \ >> > > + " >> > > SRC_URI[sha256sum] = >> > "135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d >> > 6" >> > > UPSTREAM_CHECK_REGEX = "xz-(?P<pver>\d+(\.\d+)+)\.tar" >> > > >> > > CACHED_CONFIGUREVARS += "gl_cv_posix_shell=/bin/sh" >> > > >> > > -inherit autotools gettext >> > > +inherit autotools gettext ptest >> > > >> > > PACKAGES =+ "liblzma" >> > > >> > > @@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ >> > > lzma lzcat unlzma" >> > > >> > > BBCLASSEXTEND = "native nativesdk" >> > > + >> > > +RDEPENDS:${PN}-ptest += "bash file" >> > > + >> > > +do_compile_ptest() { >> > > + oe_runmake check TESTS= >> > > +} >> > > + >> > > +do_install_ptest () { >> > > + install -d ${D}${PTEST_PATH}/tests >> > > + find ${B}/tests/.libs -type f -executable -exec cp {} >> > ${D}${PTEST_PATH}/tests \; >> > > + cp ${B}/config.h ${D}${PTEST_PATH} >> > > + for i in files xzgrep_expected_output test_files.sh test_scripts.sh >> > test_compress.sh; do >> > > + cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests >> > > + done >> > > + mkdir -p ${D}${PTEST_PATH}/src/xz >> > > + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz >> > > + mkdir -p ${D}${PTEST_PATH}/src/xzdec >> > > + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec >> > > + mkdir -p ${D}${PTEST_PATH}/src/scripts >> > > + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff >> > > + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep >> > > +} >> > > -- >> > > 2.25.1 >> > > >> > >> > > >> > > >> > > >> > >> > >> > -- >> > Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel >> > engineering https://bootlin.com >> >> >> > > > -- > Yoann Congal > Smile ECS - Tech expert > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#194917): https://lists.openembedded.org/g/openembedded-core/message/194917 > Mute This Topic: https://lists.openembedded.org/mt/103971280/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc index a1643c5548..68a3f63a6a 100644 --- a/meta/conf/distro/include/ptest-packagelists.inc +++ b/meta/conf/distro/include/ptest-packagelists.inc @@ -77,6 +77,7 @@ PTESTS_FAST = "\ sed \ slang \ wayland \ + xz \ zlib \ libexif \ " diff --git a/meta/recipes-extended/xz/xz/run-ptest b/meta/recipes-extended/xz/xz/run-ptest new file mode 100644 index 0000000000..cc8ba92016 --- /dev/null +++ b/meta/recipes-extended/xz/xz/run-ptest @@ -0,0 +1,26 @@ +#!/bin/bash + +ptestdir=$(dirname "$(readlink -f "$0")") +cd "$ptestdir"/tests || exit + +# executables test cases +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') +for i in ${tests_exec}; do + ./$i +done + +# POSIX shell script test cases +export srcdir=${ptestdir}/tests +test_shell="test_files.sh test_scripts.sh test_compress.sh" +for j in ${test_shell}; do + if [ $j == "test_compress.sh" ]; then + ./$j create_compress_files + else + ./$j + fi + if [ $? -eq 0 ]; then + echo "PASS: $j" + else + echo "FAIL: $j" + fi +done diff --git a/meta/recipes-extended/xz/xz_5.4.5.bb b/meta/recipes-extended/xz/xz_5.4.5.bb index 09eade8be0..0f0849c3e5 100644 --- a/meta/recipes-extended/xz/xz_5.4.5.bb +++ b/meta/recipes-extended/xz/xz_5.4.5.bb @@ -24,13 +24,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \ file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \ " -SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest', '', d)} \ + " SRC_URI[sha256sum] = "135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d6" UPSTREAM_CHECK_REGEX = "xz-(?P<pver>\d+(\.\d+)+)\.tar" CACHED_CONFIGUREVARS += "gl_cv_posix_shell=/bin/sh" -inherit autotools gettext +inherit autotools gettext ptest PACKAGES =+ "liblzma" @@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ lzma lzcat unlzma" BBCLASSEXTEND = "native nativesdk" + +RDEPENDS:${PN}-ptest += "bash file" + +do_compile_ptest() { + oe_runmake check TESTS= +} + +do_install_ptest () { + install -d ${D}${PTEST_PATH}/tests + find ${B}/tests/.libs -type f -executable -exec cp {} ${D}${PTEST_PATH}/tests \; + cp ${B}/config.h ${D}${PTEST_PATH} + for i in files xzgrep_expected_output test_files.sh test_scripts.sh test_compress.sh; do + cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests + done + mkdir -p ${D}${PTEST_PATH}/src/xz + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz + mkdir -p ${D}${PTEST_PATH}/src/xzdec + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec + mkdir -p ${D}${PTEST_PATH}/src/scripts + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep +}
There are two types of cases: executables and POSIX shell scripts. All test cases PASS. Add xz-ptest to PTESTS_FAST because test duration less than 30s on qemux86-64. root@qemux86-64:~# ptest-runner xz START: ptest-runner 2024-01-26T03:32 BEGIN: /usr/lib/xz/ptest === test_bcj_exact_size.c === PASS: test_exact_size PASS: test_empty_block --- === END === ... === test_vli.c === PASS: test_lzma_vli_size PASS: test_lzma_vli_encode PASS: test_lzma_vli_decode --- === END === PASS: test_files.sh PASS: test_scripts.sh PASS: test_compress.sh DURATION: 18 END: /usr/lib/xz/ptest 2024-01-26T03:33 STOP: ptest-runner TOTAL: 1 FAIL: 0 Signed-off-by: Xu, Chi <chi.xu@windriver.com> --- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ meta/recipes-extended/xz/xz_5.4.5.bb | 28 +++++++++++++++++-- 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-extended/xz/xz/run-ptest