Message ID | 20241022081140.1464860-1-thakur.virendra1810@gmail.com |
---|---|
State | New |
Headers | show |
Series | [master] libyaml: Add libyaml ptest | expand |
On Tue, 22 Oct 2024 at 10:13, virendra thakur via lists.openembedded.org <thakur.virendra1810=gmail.com@lists.openembedded.org> wrote: > +++ b/meta/recipes-support/libyaml/libyaml/run-ptest > @@ -0,0 +1,28 @@ > +#!/bin/sh > +for testlist in test-version test-reader run-emitter run-dumper run-parser run-scanner run-loader; do > + case ${testlist} in > + test-version|test-reader) > + outputrun=$(./$testlist) > + if ! echo ${outputrun}| grep -q "FAILURE" > + then > + echo "PASS: ${testlist}" > + else > + echo "FAIL : ${testlist}" > + exit 1 > + fi > + ;; > + esac > + case ${testlist} in > + run-emitter|run-dumper|run-parser|run-scanner|run-loader) > + outputrun=$(./$testlist example.yaml) > + echo $outputrun > + if ! echo ${outputrun}| grep "FAILURE ([1-50]" > + then > + echo "PASS: ${testlist}" > + else > + echo "FAIL : ${testlist}" > + exit 1 > + fi > + ;; > + esac > +done I'd like to better understand what is happening here. Why are some items in the test suite run without an argument, and other items with example.yaml that we created ourselves (apparently, from items in examples/)? How does the test suite work upstream and why can't we simply use that? > SRC_URI[md5sum] = "bb15429d8fb787e7d3f1c83ae129a999" > SRC_URI[sha256sum] = "c642ae9b75fee120b2d96c712538bd2cf283228d2337df2cf2988e3c02678ef4" > +SRC_URI[run-ptest.md5sum] = "040b47d9b00d2ddbc707792ff6626243" > +SRC_URI[run-ptest.sha256sum] = "71030494ab116f1c31bf125de9d5fbe2f01aacf4daada36b44202be135a80964" > +SRC_URI[example.yaml.md5sum] = "d7f8a342650b153d219431633b0c383b" > +SRC_URI[example.yaml.sha256sum] = "d7c50b29716fcc7445e181c7d7043d70f8e58096793dc1f351c8e7f41034f813" This is not necessary. Integrity check for files in the layer tree is ensured through git commit id (of the layer) being a checksum of all the files. Alex
Below test does not require any argument to be pass. Its is a simple binary > + test-version and test-reader run-emitter|run-dumper|run-parser|run-scanner|run-loader Other test require an argument and yaml file which it needs to perform task. Like parsing or emmiting or to perform run-loader test I will send another version of patch without below change: > +SRC_URI[run-ptest.md5sum] = "040b47d9b00d2ddbc707792ff6626243" > +SRC_URI[run-ptest.sha256sum] = "71030494ab116f1c31bf125de9d5fbe2f01aacf4daada36b44202be135a80964" > +SRC_URI[example.yaml.md5sum] = "d7f8a342650b153d219431633b0c383b" > +SRC_URI[example.yaml.sha256sum] = "d7c50b29716fcc7445e181c7d7043d70f8e58096793dc1f351c8e7f41034f813" [cid:95654072-e768-4291-9780-247df4bf5f3a]<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature> Book time to meet with me<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature>
On Tue, 19 Nov 2024 at 13:02, Virendra Thakur via lists.openembedded.org <virendra.thakur=kpit.com@lists.openembedded.org> wrote: > > > Below test does not require any argument to be pass. Its is a simple binary > > + test-version and test-reader > > > run-emitter|run-dumper|run-parser|run-scanner|run-loader > Other test require an argument and yaml file which it needs to perform > task. Like parsing or emmiting or to perform run-loader test > This does not quite answer this part of my question: How does the test suite work upstream and why can't we simply use that? I still don't understand why a custom yaml needs to be created. Alex
in yocto , Libyaml are sourced from https://pyyaml.org/download/libyaml/yaml-0.2.5.tar.gz when I tried to run the test using above source . It only execute two test binary. yaml-0.2.5/tests$ make check-TESTS make[1]: Entering directory 'yaml-0.2.5/tests' PASS: test-version PASS: test-reader ============================================================================ Testsuite summary for yaml 0.2.5 ============================================================================ # TOTAL: 2 # PASS: 2 # SKIP: 0 # XFAIL: 0 # FAIL: 0 # XPASS: 0 # ERROR: 0 ============================================================================ make[1]: Leaving directory yaml-0.2.5/tests' So added few more test binary and an example yaml file as test input. So below are the test binary . run-emitter run-dumper run-parser run-scanner run-loader [cid:6e137582-c02d-4e71-a295-5ee18e5b2928]<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature> Book time to meet with me<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature>
On Tue, 19 Nov 2024 at 14:24, Virendra Kumar Thakur < Virendra.Thakur@kpit.com> wrote: > in yocto , Libyaml are sourced from > https://pyyaml.org/download/libyaml/yaml-0.2.5.tar.gz > > when I tried to run the test using above source . It only execute two test > binary. > > yaml-0.2.5/tests$ make check-TESTS > make[1]: Entering directory 'yaml-0.2.5/tests' > PASS: test-version > PASS: test-reader > If you take a look at upstream's top level Makefile.am, it has an additional target: ============= test-suite: tests/run-test-suite all make -C $< test test-all: test test-suite tests/run-test-suite: ifeq ($(LIBYAML_TEST_SUITE_RUN_REPO),$(LIBYAML_TEST_SUITE_RUN_REPO_DEFAULT)) -git branch --track $(LIBYAML_TEST_SUITE_RUN_BRANCH) origin/$(LIBYAML_TEST_SUITE_RUN_BRANCH) -git worktree prune git worktree add $@ $(LIBYAML_TEST_SUITE_RUN_BRANCH) else git clone --branch $(LIBYAML_TEST_SUITE_RUN_BRANCH) $(LIBYAML_TEST_SUITE_RUN_REPO) $@ endif ============ Presumably, running 'make test-all' will fetch the needed test data from somewhere on the internet, and then use it to run additional tests. Can you please try that, and see what happens? Then we can probably just use the data from that repo by putting it into SRC_URI. Alex
Hello @alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com> I have tried to run test-all but it fails: make[2]: Entering directory '/home/virendrak/yaml-0.2.5/tests' PASS: test-version PASS: test-reader ============================================================================ Testsuite summary for yaml 0.2.5 ============================================================================ # TOTAL: 2 # PASS: 2 # SKIP: 0 # XFAIL: 0 # FAIL: 0 # XPASS: 0 # ERROR: 0 ============================================================================ make[2]: Leaving directory '/home/virendrak/yaml-0.2.5/tests' make[1]: Leaving directory '/home/virendrak/yaml-0.2.5/tests' git branch --track run-test-suite origin/run-test-suite fatal: not a git repository (or any of the parent directories): .git make: [Makefile:885: tests/run-test-suite] Error 128 (ignored) git worktree prune fatal: not a git repository (or any of the parent directories): .git make: [Makefile:886: tests/run-test-suite] Error 128 (ignored) git worktree add tests/run-test-suite run-test-suite fatal: not a git repository (or any of the parent directories): .git make: *** [Makefile:887: tests/run-test-suite] Error 128 It fails due to libyaml source is from downloaded from dowload directory mentioned below as a tar , its not git repo. In yocto libyaml SRC is from : SRC_URI = "https://pyyaml.org/download/libyaml/yaml-${PV}.tar.gz" Due to this it fail to fetch code from git repo. Best Regards [cid:8dc09906-5365-4dd5-b686-c6a3e5ee9ead]<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature> Book time to meet with me<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature>
On Fri, 22 Nov 2024 at 13:38, Virendra Kumar Thakur < Virendra.Thakur@kpit.com> wrote: > It fails due to libyaml source is from downloaded from dowload directory > mentioned below as a tar , its not git repo. > In yocto libyaml SRC is from : SRC_URI = " > https://pyyaml.org/download/libyaml/yaml-${PV}.tar.gz" > > Due to this it fail to fetch code from git repo. > For the sake of experiment, perhaps you can try again with a git checkout to 0.2.5 tag from here? https://github.com/yaml/libyaml Alex
Hello alex With code from github it works fine. make test-all is working. I am attaching here with the output file. 1..214 ok All tests successful. Files=3, Tests=452, 12 wallclock secs ( 0.12 usr 0.03 sys + 3.22 cusr 5.92 csys = 9.29 CPU) Result: PASS [cid:dc67cc71-2784-492b-bad5-d29bf68ca91d]<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature> Book time to meet with me<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature>
On Fri, 22 Nov 2024 at 14:00, Virendra Kumar Thakur <Virendra.Thakur@kpit.com> wrote: > With code from github it works fine. make test-all is working. > > I am attaching here with the output file. > > 1..214 > ok > All tests successful. > Files=3, Tests=452, 12 wallclock secs ( 0.12 usr 0.03 sys + 3.22 cusr 5.92 csys = 9.29 CPU) > Result: PASS Right. Then your next task is to figure out where those tests obtain the needed data, and then replicate that in oe-core, possibly by adding needed repositories to SRC_URI, and then copying bits of them into the ptest package. The key sequence in the log seems to be this: git branch --track run-test-suite origin/run-test-suite Branch 'run-test-suite' set up to track remote branch 'run-test-suite' from 'origin'. git worktree prune git worktree add tests/run-test-suite run-test-suite Preparing worktree (checking out 'run-test-suite') HEAD is now at cc9cb7b More bin/pin debug messages make -C tests/run-test-suite test make[1]: Entering directory '/home/virendrak/libyaml-github/libyaml/tests/run-test-suite' git clone https://github.com/yaml/yaml-test-suite data --branch=data Cloning into 'data'... remote: Enumerating objects: 17410, done. remote: Counting objects: 100% (1134/1134), done. remote: Compressing objects: 100% (555/555), done. remote: Total 17410 (delta 675), reused 580 (delta 579), pack-reused 16276 (from 1) Receiving objects: 100% (17410/17410), 4.57 MiB | 9.07 MiB/s, done. Resolving deltas: 100% (11215/11215), done. (cd data && git reset --hard data-2020-02-11) HEAD is now at a12ee724 Regenerated data from master v2020-02-11 git branch --track run-test-suite-code origin/run-test-suite-code Branch 'run-test-suite-code' set up to track remote branch 'run-test-suite-code' from 'origin'. git worktree prune [[ -d test ]] || \ git worktree add test run-test-suite-code Preparing worktree (checking out 'run-test-suite-code') HEAD is now at 67365dd Remove tests for PR#162 (cd test && git reset --hard 67365dd2af265f17ba6ee0832c1411e4fd80ac0b) HEAD is now at 67365dd Remove tests for PR#162 prove -lv test Alex
Hello Alexander, Test which I have added is addressing our purpose. Not working on this topic now. I will work on your suggestion If i will get some time. [cid:8de35c11-2683-4bee-816d-79b5c95da7c9]<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature> Book time to meet with me<https://outlook.office.com/bookwithme/user/fcab7d9903984b91a5841c51f49bd5ec@kpit.com?anonymous&ep=bwmEmailSignature>
On Wed, 27 Nov 2024 at 14:39, Virendra Kumar Thakur < Virendra.Thakur@kpit.com> wrote: > Test which I have added is addressing our purpose. > Not working on this topic now. I will work on your suggestion If i will > get some time. > The problem is that the test adds custom test data that isn't possible to independently recreate (as the only information about it is that it was 'created by using examples'). This makes it very difficult to do later adjustments to that data when needed (e.g. when upstream changes the tests that start causing failures with the data). On the other hand, upstream does provide official test data, and that's what we should be using. For this reason I cannot accept the implementation as it is. Alex
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc index 4a48188562..cd9fae3e45 100644 --- a/meta/conf/distro/include/ptest-packagelists.inc +++ b/meta/conf/distro/include/ptest-packagelists.inc @@ -48,6 +48,7 @@ PTESTS_FAST = "\ libxml-simple-perl \ libxml2 \ libxmlb \ + libyaml \ logrotate \ lua \ lzo \ diff --git a/meta/recipes-support/libyaml/libyaml/example.yaml b/meta/recipes-support/libyaml/libyaml/example.yaml new file mode 100644 index 0000000000..107b2b1204 --- /dev/null +++ b/meta/recipes-support/libyaml/libyaml/example.yaml @@ -0,0 +1,45 @@ +# anchors yaml +base: &base + name: Everyone has same name + +foo: &foo + <<: *base + age: 10 + +bar: &bar + <<: *base + age: 20 +# global tag example +%TAG ! tag:clarkevans.com,2002: +--- !shape + # Use the ! handle for presenting + # tag:clarkevans.com,2002:circle +- !circle + center: &ORIGIN {x: 73, y: 129} + radius: 7 +- !line + start: *ORIGIN + finish: { x: 89, y: 102 } +- !label + start: *ORIGIN + color: 0xFFEEBB + text: Pretty vector drawing. +# array example +- member +- member2 +# numbers example +--- +string: "Hello" +integer: 123 +float: 12.345 +boolean: No +# strings example +message: "YAML & JSON" +# mapping example +person: + name: "ramesh" + surname: "sharma" + age: 23 +#json example +--- +{"key": ["value", 3]} diff --git a/meta/recipes-support/libyaml/libyaml/run-ptest b/meta/recipes-support/libyaml/libyaml/run-ptest new file mode 100644 index 0000000000..794b24c6ba --- /dev/null +++ b/meta/recipes-support/libyaml/libyaml/run-ptest @@ -0,0 +1,28 @@ +#!/bin/sh +for testlist in test-version test-reader run-emitter run-dumper run-parser run-scanner run-loader; do + case ${testlist} in + test-version|test-reader) + outputrun=$(./$testlist) + if ! echo ${outputrun}| grep -q "FAILURE" + then + echo "PASS: ${testlist}" + else + echo "FAIL : ${testlist}" + exit 1 + fi + ;; + esac + case ${testlist} in + run-emitter|run-dumper|run-parser|run-scanner|run-loader) + outputrun=$(./$testlist example.yaml) + echo $outputrun + if ! echo ${outputrun}| grep "FAILURE ([1-50]" + then + echo "PASS: ${testlist}" + else + echo "FAIL : ${testlist}" + exit 1 + fi + ;; + esac +done diff --git a/meta/recipes-support/libyaml/libyaml_0.2.5.bb b/meta/recipes-support/libyaml/libyaml_0.2.5.bb index aa7fc5e914..7b108b8548 100644 --- a/meta/recipes-support/libyaml/libyaml_0.2.5.bb +++ b/meta/recipes-support/libyaml/libyaml_0.2.5.bb @@ -7,13 +7,34 @@ SECTION = "libs/devel" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://License;md5=7bbd28caa69f81f5cd5f48647236663d" -SRC_URI = "https://pyyaml.org/download/libyaml/yaml-${PV}.tar.gz" +SRC_URI = "https://pyyaml.org/download/libyaml/yaml-${PV}.tar.gz \ + file://run-ptest \ + file://example.yaml \ + " SRC_URI[md5sum] = "bb15429d8fb787e7d3f1c83ae129a999" SRC_URI[sha256sum] = "c642ae9b75fee120b2d96c712538bd2cf283228d2337df2cf2988e3c02678ef4" +SRC_URI[run-ptest.md5sum] = "040b47d9b00d2ddbc707792ff6626243" +SRC_URI[run-ptest.sha256sum] = "71030494ab116f1c31bf125de9d5fbe2f01aacf4daada36b44202be135a80964" +SRC_URI[example.yaml.md5sum] = "d7f8a342650b153d219431633b0c383b" +SRC_URI[example.yaml.sha256sum] = "d7c50b29716fcc7445e181c7d7043d70f8e58096793dc1f351c8e7f41034f813" S = "${WORKDIR}/yaml-${PV}" -inherit autotools +inherit autotools ptest + +do_compile_ptest() { + sed -i "/\$(MAKE) \$(AM_MAKEFLAGS) check-TESTS/d" tests/Makefile + oe_runmake check +} + +do_install_ptest() { + # install the tests dir + cp -rf ${B}/tests/.libs/* ${D}${PTEST_PATH} + install -m 755 ${UNPACKDIR}/example.yaml ${D}${PTEST_PATH} + +} + +RDEPENDS:${PN}-ptest += "make bash" DISABLE_STATIC:class-nativesdk = "" DISABLE_STATIC:class-native = ""
Install libyaml test suite and run it as ptest. The run-ptest script iterates through executable files copied from the host, runs each on the target. example.yaml file created by using examples present in libyaml upstream. Reference: https://github.com/yaml/libyaml/tree/master/examples Signed-off-by: Virendra Thakur <thakur.virendra1810@gmail.com> --- .../distro/include/ptest-packagelists.inc | 1 + .../libyaml/libyaml/example.yaml | 45 +++++++++++++++++++ .../recipes-support/libyaml/libyaml/run-ptest | 28 ++++++++++++ meta/recipes-support/libyaml/libyaml_0.2.5.bb | 25 ++++++++++- 4 files changed, 97 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-support/libyaml/libyaml/example.yaml create mode 100644 meta/recipes-support/libyaml/libyaml/run-ptest