| Message ID | 20221104100140.976337-1-chase.qi@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] libc-test: add libc testsuite for musl | expand |
On Fri, Nov 04, 2022 at 10:01:40AM +0000, Chase Qi wrote: > libc-test is a collection of unit test to measure the correctness and > robustness of a C/POSIX standard library implementation. It is developed > as part of the musl project. > > Signed-off-by: Chase Qi <chase.qi@linaro.org> > --- > .../distro/include/ptest-packagelists.inc | 1 + > meta/recipes-core/musl/libc-test/run-ptest | 17 +++++++ > meta/recipes-core/musl/libc-test_git.bb | 50 +++++++++++++++++++ > 3 files changed, 68 insertions(+) > create mode 100644 meta/recipes-core/musl/libc-test/run-ptest > create mode 100644 meta/recipes-core/musl/libc-test_git.bb > > diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc > index 32b0e5297a..3c9ff43bd9 100644 > --- a/meta/conf/distro/include/ptest-packagelists.inc > +++ b/meta/conf/distro/include/ptest-packagelists.inc > @@ -105,6 +105,7 @@ PTESTS_SLOW = "\ > > PTESTS_SLOW:remove:riscv64 = "valgrind-ptest" > PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest" > +PTESTS_SLOW:append:libc-musl = " libc-test-ptest" > > # ruby-ptest \ # Timeout > # lz4-ptest \ # Needs a rewrite > diff --git a/meta/recipes-core/musl/libc-test/run-ptest b/meta/recipes-core/musl/libc-test/run-ptest > new file mode 100644 > index 0000000000..a30cee590e > --- /dev/null > +++ b/meta/recipes-core/musl/libc-test/run-ptest > @@ -0,0 +1,17 @@ > +#!/bin/sh > + Please use at least "set -e" to capture errors of failing commands. > +cd /opt/libc-test > +make cleanall > +make run | tee libc-test.log Depending how this handles errors, the return value from "make run" may need to be discarded, but I'd prefer this to be explicit if "set -e" is used. > +echo "" > +echo "--- ptest result ---" > +if grep -q '^FAIL src.*\.exe.*' libc-test.log; then > + # Consolidate output format. > + # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main" > + grep '^FAIL src.*\.exe.*' libc-test.log \ > + | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||' > + exit 1 > +else > + echo "PASS: libc-test" Would be nice if something would check that tests were actually executed. Cheers, -Mikko
On 4 Nov 2022, at 10:01, Chase Qi via lists.openembedded.org <chase.qi=linaro.org@lists.openembedded.org> wrote: > > +SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398" > +SRC_URI = " \ > + git://nsz.repo.hu:49100/repo/libc-test;branch=master \ > + file://run-ptest \ > +" > +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test” The project’s web site at https://wiki.musl-libc.org/libc-test.html says the git URL is git://repo.or.cz/libc-test, so that’s the URL we should fetch from. Also as this project doesn’t appear to have versions, set: PV=“0+git${SRCPV}”. This way the version has the git SHA in, and if they ever release a version it will be greater than 0 and sort correctly. Ross
Hi Mikko, Thanks for the review. On Fri, 4 Nov 2022 at 18:20, Mikko Rapeli <mikko.rapeli@linaro.org> wrote: > > On Fri, Nov 04, 2022 at 10:01:40AM +0000, Chase Qi wrote: > > libc-test is a collection of unit test to measure the correctness and > > robustness of a C/POSIX standard library implementation. It is developed > > as part of the musl project. > > > > Signed-off-by: Chase Qi <chase.qi@linaro.org> > > --- > > .../distro/include/ptest-packagelists.inc | 1 + > > meta/recipes-core/musl/libc-test/run-ptest | 17 +++++++ > > meta/recipes-core/musl/libc-test_git.bb | 50 +++++++++++++++++++ > > 3 files changed, 68 insertions(+) > > create mode 100644 meta/recipes-core/musl/libc-test/run-ptest > > create mode 100644 meta/recipes-core/musl/libc-test_git.bb > > > > diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc > > index 32b0e5297a..3c9ff43bd9 100644 > > --- a/meta/conf/distro/include/ptest-packagelists.inc > > +++ b/meta/conf/distro/include/ptest-packagelists.inc > > @@ -105,6 +105,7 @@ PTESTS_SLOW = "\ > > > > PTESTS_SLOW:remove:riscv64 = "valgrind-ptest" > > PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest" > > +PTESTS_SLOW:append:libc-musl = " libc-test-ptest" > > > > # ruby-ptest \ # Timeout > > # lz4-ptest \ # Needs a rewrite > > diff --git a/meta/recipes-core/musl/libc-test/run-ptest b/meta/recipes-core/musl/libc-test/run-ptest > > new file mode 100644 > > index 0000000000..a30cee590e > > --- /dev/null > > +++ b/meta/recipes-core/musl/libc-test/run-ptest > > @@ -0,0 +1,17 @@ > > +#!/bin/sh > > + > > Please use at least "set -e" to capture errors of failing > commands. > +1 Enabled in v3. > > +cd /opt/libc-test > > +make cleanall > > +make run | tee libc-test.log > > Depending how this handles errors, the return value from > "make run" may need to be discarded, but I'd prefer > this to be explicit if "set -e" is used. > Yes. In case the command return with non-zero, I switched to "make run || true" in v3 for the following log parsing. > > +echo "" > > +echo "--- ptest result ---" > > +if grep -q '^FAIL src.*\.exe.*' libc-test.log; then > > + # Consolidate output format. > > + # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main" > > + grep '^FAIL src.*\.exe.*' libc-test.log \ > > + | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||' > > + exit 1 > > +else > > + echo "PASS: libc-test" > > Would be nice if something would check that tests were actually > executed. I think src/REPORT can be used to check if 'make run' is finished correctly. Per the logic here http://nsz.repo.hu/git/?p=libc-test;a=blob;f=Makefile;h=cf2ab2b2c8ebef0a9eab8cb06efc7bb5d989cc83;hb=HEAD#l120, I understand the file is generated after all sub-tests are finished. Would you please review v3? Thanks, Chase > > Cheers, > > -Mikko
Hi Ross, Thanks for the review. On Fri, 4 Nov 2022 at 22:13, Ross Burton <Ross.Burton@arm.com> wrote: > > On 4 Nov 2022, at 10:01, Chase Qi via lists.openembedded.org <chase.qi=linaro.org@lists.openembedded.org> wrote: > > > > +SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398" > > +SRC_URI = " \ > > + git://nsz.repo.hu:49100/repo/libc-test;branch=master \ > > + file://run-ptest \ > > +" > > +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test” > > The project’s web site at https://wiki.musl-libc.org/libc-test.html says the git URL is git://repo.or.cz/libc-test, so that’s the URL we should fetch from. > The summary page https://repo.or.cz/libc-test.git shows it is a mirror URL of git://nsz.repo.hu:49100/repo/libc-test. In v3, I removed the MIRRORS line. https://repo.or.cz/libc-test.git is used directly now. At least, there is a team maintaining the services. > Also as this project doesn’t appear to have versions, set: > > PV=“0+git${SRCPV}”. > > This way the version has the git SHA in, and if they ever release a version it will be greater than 0 and sort correctly. Make sense, added in v3. Thanks for the suggestion. - Chase > > Ross
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc index 32b0e5297a..3c9ff43bd9 100644 --- a/meta/conf/distro/include/ptest-packagelists.inc +++ b/meta/conf/distro/include/ptest-packagelists.inc @@ -105,6 +105,7 @@ PTESTS_SLOW = "\ PTESTS_SLOW:remove:riscv64 = "valgrind-ptest" PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest" +PTESTS_SLOW:append:libc-musl = " libc-test-ptest" # ruby-ptest \ # Timeout # lz4-ptest \ # Needs a rewrite diff --git a/meta/recipes-core/musl/libc-test/run-ptest b/meta/recipes-core/musl/libc-test/run-ptest new file mode 100644 index 0000000000..a30cee590e --- /dev/null +++ b/meta/recipes-core/musl/libc-test/run-ptest @@ -0,0 +1,17 @@ +#!/bin/sh + +cd /opt/libc-test +make cleanall +make run | tee libc-test.log + +echo "" +echo "--- ptest result ---" +if grep -q '^FAIL src.*\.exe.*' libc-test.log; then + # Consolidate output format. + # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main" + grep '^FAIL src.*\.exe.*' libc-test.log \ + | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||' + exit 1 +else + echo "PASS: libc-test" +fi diff --git a/meta/recipes-core/musl/libc-test_git.bb b/meta/recipes-core/musl/libc-test_git.bb new file mode 100644 index 0000000000..091aec74fd --- /dev/null +++ b/meta/recipes-core/musl/libc-test_git.bb @@ -0,0 +1,50 @@ +SUMMARY = "Musl libc unit tests" +HOMEPAGE = "https://wiki.musl-libc.org/libc-test.html" +DESCRIPTION = "libc-test is a collection of unit tests to measure the \ +correctness and robustness of a C/POSIX standard library implementation. It is \ +developed as part of the musl project." +SECTION = "tests" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=43ed1245085be90dc934288117d55a3b" + +inherit ptest + +SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398" +SRC_URI = " \ + git://nsz.repo.hu:49100/repo/libc-test;branch=master \ + file://run-ptest \ +" +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test" + +S = "${WORKDIR}/git" + +# libc-test 'make' or 'make run' command is designed to build and run tests. It +# reports both build and test failures. The commands should be run on target. +do_compile() { + : +} + +RDEPENDS:${PN} = " \ + bash \ + grep \ + musl \ + packagegroup-core-buildessential \ +" + + RDEPENDS:${PN}-ptest = " \ + ${PN} \ + sed \ +" + +install_path = "/opt/${PN}" +FILES:${PN} += "${install_path}/*" + +do_install () { + install -d ${D}${install_path}/ + cp ${S}/Makefile ${D}${install_path} + cp ${S}/config.mak.def ${D}${install_path}/config.mak + cp -r ${S}/src ${D}${install_path} +} + +COMPATIBLE_HOST = "null" +COMPATIBLE_HOST:libc-musl = "(.*)"
libc-test is a collection of unit test to measure the correctness and robustness of a C/POSIX standard library implementation. It is developed as part of the musl project. Signed-off-by: Chase Qi <chase.qi@linaro.org> --- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-core/musl/libc-test/run-ptest | 17 +++++++ meta/recipes-core/musl/libc-test_git.bb | 50 +++++++++++++++++++ 3 files changed, 68 insertions(+) create mode 100644 meta/recipes-core/musl/libc-test/run-ptest create mode 100644 meta/recipes-core/musl/libc-test_git.bb