Message ID | 20221103053926.719998-1-chase.qi@linaro.org |
---|---|
State | New |
Headers | show |
Series | libc-test: add libc testsuite for musl | expand |
Hello, On 03/11/2022 05:39:26+0000, Chase Qi wrote: > libc-test is a collection of unit test to measure the correctness and > robustness of a C/POSIX standard library implementation. It is developed > as part of the musl project. > You also need to add it to the ptest list: WARNING: libc-test-git-r0 do_package_qa: QA Issue: supports ptests but is not included in oe-core's ptest-packagelists.inc [missing-ptest] > Signed-off-by: Chase Qi <chase.qi@linaro.org> > --- > meta/recipes-core/musl/libc-test/run-ptest | 17 ++++++++ > meta/recipes-core/musl/libc-test_git.bb | 45 ++++++++++++++++++++++ > 2 files changed, 62 insertions(+) > create mode 100644 meta/recipes-core/musl/libc-test/run-ptest > create mode 100644 meta/recipes-core/musl/libc-test_git.bb > > diff --git a/meta/recipes-core/musl/libc-test/run-ptest b/meta/recipes-core/musl/libc-test/run-ptest > new file mode 100644 > index 0000000000..30abcdfba5 > --- /dev/null > +++ b/meta/recipes-core/musl/libc-test/run-ptest > @@ -0,0 +1,17 @@ > +#!/bin/sh > + > +make cleanall > +make run | tee libc-test.log > + > +echo "" > +echo "--- ptest result ---" > +if grep -q '^FAIL src.*\.exe.*' libc-test.log; then > + # Consolidate output format. > + # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main" > + grep '^FAIL src.*\.exe.*' libc-test.log \ > + | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||' > + exit 1 > +else > + echo "PASS: libc-test" > + exit 0 > +fi > diff --git a/meta/recipes-core/musl/libc-test_git.bb b/meta/recipes-core/musl/libc-test_git.bb > new file mode 100644 > index 0000000000..d5d8845727 > --- /dev/null > +++ b/meta/recipes-core/musl/libc-test_git.bb > @@ -0,0 +1,45 @@ > +SUMMARY = "Musl libc test to be run with ptest" > +HOMEPAGE = "https://wiki.musl-libc.org/libc-test.html" > +DESCRIPTION = "libc-test is a collection of unit tests to measure the \ > +correctness and robustness of a C/POSIX standard library implementation. It is \ > +developed as part of the musl project." > +SECTION = "tests" > +LICENSE = "MIT" > +LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=43ed1245085be90dc934288117d55a3b" > + > +inherit ptest > + > +SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398" > +SRC_URI = " \ > + git://nsz.repo.hu:49100/repo/libc-test;branch=master \ > + file://run-ptest \ > +" > +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test" > + > +S = "${WORKDIR}/git" > + > +# libc-test 'make' or 'make run' command is designed to build and run tests. It > +# reports both build and test failures. The commands should be run on target. > +do_compile() { > + : > +} > + > +RDEPENDS:${PN}-ptest = " \ > + bash \ > + grep \ > + musl \ > + packagegroup-core-buildessential \ > + sed \ > +" > + > +do_install_ptest () { > + cp ${S}/Makefile ${D}${PTEST_PATH} > + cp ${S}/config.mak.def ${D}${PTEST_PATH}/config.mak > + cp -r ${S}/src ${D}${PTEST_PATH}/ > +} > + > +# libc-test-ptest depends on empty libc-test. > +ALLOW_EMPTY:${PN} = "1" > + > +COMPATIBLE_HOST = "null" > +COMPATIBLE_HOST:libc-musl = "(.*)" > -- > 2.25.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#172610): https://lists.openembedded.org/g/openembedded-core/message/172610 > Mute This Topic: https://lists.openembedded.org/mt/94751275/3617179 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On 3 Nov 2022, at 05:39, Chase Qi via lists.openembedded.org <chase.qi=linaro.org@lists.openembedded.org> wrote: > +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test" Why do this recipe need a MIRRORS? > +RDEPENDS:${PN}-ptest = " \ > + bash \ > + grep \ > + musl \ > + packagegroup-core-buildessential \ > + sed \ > +" > + > +do_install_ptest () { > + cp ${S}/Makefile ${D}${PTEST_PATH} > + cp ${S}/config.mak.def ${D}${PTEST_PATH}/config.mak > + cp -r ${S}/src ${D}${PTEST_PATH}/ > +} > + > +# libc-test-ptest depends on empty libc-test. > +ALLOW_EMPTY:${PN} = "1" It would be neater if the test suite was installed to libc-test, and libc-test-ptest simply contained the run-ptest script. This way the packaging reflects reality and you don’t need to set ALLOW_EMPTY. Basically: the package contains a test suite, and a ptest harness. If we ever move from ptest to something else then the libc-test package won’t change, just the harness will. > +COMPATIBLE_HOST = "null" > +COMPATIBLE_HOST:libc-musl = "(.*)" Is there any value in running the test suite on glibc? Ross
Hi Ross, Thanks for the review. On Thu, 3 Nov 2022 at 21:51, Ross Burton <Ross.Burton@arm.com> wrote: > > On 3 Nov 2022, at 05:39, Chase Qi via lists.openembedded.org <chase.qi=linaro.org@lists.openembedded.org> wrote: > > +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test" > > Why do this recipe need a MIRRORS? > The main repo looks like a personal repo hosted in a kind of self-maintained hosting service. It is owned by http://igor2.repo.hu/. The mirror is the official mirror documented on the wiki https://wiki.musl-libc.org/libc-test.html. I think it can be helpful when the main repo is unreachable. > > +RDEPENDS:${PN}-ptest = " \ > > + bash \ > > + grep \ > > + musl \ > > + packagegroup-core-buildessential \ > > + sed \ > > +" > > + > > +do_install_ptest () { > > + cp ${S}/Makefile ${D}${PTEST_PATH} > > + cp ${S}/config.mak.def ${D}${PTEST_PATH}/config.mak > > + cp -r ${S}/src ${D}${PTEST_PATH}/ > > +} > > + > > +# libc-test-ptest depends on empty libc-test. > > +ALLOW_EMPTY:${PN} = "1" > > It would be neater if the test suite was installed to libc-test, and libc-test-ptest simply contained the run-ptest script. This way the packaging reflects reality and you don’t need to set ALLOW_EMPTY. > > Basically: the package contains a test suite, and a ptest harness. If we ever move from ptest to something else then the libc-test package won’t change, just the harness will. > +1 Thanks for your proposal. I have submitted v2. Would you please take a look? > > +COMPATIBLE_HOST = "null" > > +COMPATIBLE_HOST:libc-musl = "(.*)" > > Is there any value in running the test suite on glibc? > I tried the test suite on glibc. It reported all tests passed, but I do see errors in the test log. I think it doesn't work well on glibc. Considering glibc has its own test suite, the 'meta/recipes-core/glibc/glibc-tests_2.36.bb', I think we should start with musl only. Thanks, Chase > Ross >
diff --git a/meta/recipes-core/musl/libc-test/run-ptest b/meta/recipes-core/musl/libc-test/run-ptest new file mode 100644 index 0000000000..30abcdfba5 --- /dev/null +++ b/meta/recipes-core/musl/libc-test/run-ptest @@ -0,0 +1,17 @@ +#!/bin/sh + +make cleanall +make run | tee libc-test.log + +echo "" +echo "--- ptest result ---" +if grep -q '^FAIL src.*\.exe.*' libc-test.log; then + # Consolidate output format. + # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main" + grep '^FAIL src.*\.exe.*' libc-test.log \ + | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||' + exit 1 +else + echo "PASS: libc-test" + exit 0 +fi diff --git a/meta/recipes-core/musl/libc-test_git.bb b/meta/recipes-core/musl/libc-test_git.bb new file mode 100644 index 0000000000..d5d8845727 --- /dev/null +++ b/meta/recipes-core/musl/libc-test_git.bb @@ -0,0 +1,45 @@ +SUMMARY = "Musl libc test to be run with ptest" +HOMEPAGE = "https://wiki.musl-libc.org/libc-test.html" +DESCRIPTION = "libc-test is a collection of unit tests to measure the \ +correctness and robustness of a C/POSIX standard library implementation. It is \ +developed as part of the musl project." +SECTION = "tests" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=43ed1245085be90dc934288117d55a3b" + +inherit ptest + +SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398" +SRC_URI = " \ + git://nsz.repo.hu:49100/repo/libc-test;branch=master \ + file://run-ptest \ +" +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test" + +S = "${WORKDIR}/git" + +# libc-test 'make' or 'make run' command is designed to build and run tests. It +# reports both build and test failures. The commands should be run on target. +do_compile() { + : +} + +RDEPENDS:${PN}-ptest = " \ + bash \ + grep \ + musl \ + packagegroup-core-buildessential \ + sed \ +" + +do_install_ptest () { + cp ${S}/Makefile ${D}${PTEST_PATH} + cp ${S}/config.mak.def ${D}${PTEST_PATH}/config.mak + cp -r ${S}/src ${D}${PTEST_PATH}/ +} + +# libc-test-ptest depends on empty libc-test. +ALLOW_EMPTY:${PN} = "1" + +COMPATIBLE_HOST = "null" +COMPATIBLE_HOST:libc-musl = "(.*)"
libc-test is a collection of unit test to measure the correctness and robustness of a C/POSIX standard library implementation. It is developed as part of the musl project. Signed-off-by: Chase Qi <chase.qi@linaro.org> --- meta/recipes-core/musl/libc-test/run-ptest | 17 ++++++++ meta/recipes-core/musl/libc-test_git.bb | 45 ++++++++++++++++++++++ 2 files changed, 62 insertions(+) create mode 100644 meta/recipes-core/musl/libc-test/run-ptest create mode 100644 meta/recipes-core/musl/libc-test_git.bb