| Message ID | 20240902073918.490722-1-Deepthi.Hemraj@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | gcc-sanitizers: Add ptest support for AddressSanitizer tests | expand |
Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information: --- Testing patch /home/patchtest/share/mboxes/gcc-sanitizers-Add-ptest-support-for-AddressSanitizer-tests.patch FAIL: test Signed-off-by presence: A patch file has been added without a Signed-off-by tag: 'asan-exp.patch' (test_patch.TestPatch.test_signed_off_by_presence) PASS: test CVE tag format (test_patch.TestPatch.test_cve_tag_format) PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence) PASS: test Upstream-Status presence (test_patch.TestPatch.test_upstream_status_presence_format) PASS: test author valid (test_mbox.TestMbox.test_author_valid) PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format) PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence) PASS: test max line length (test_metadata.TestMetadata.test_max_line_length) PASS: test mbox format (test_mbox.TestMbox.test_mbox_format) PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade) PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format) PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length) SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint) SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files) SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore) SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned) SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence) SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence) SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint) SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head) SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files) SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence) SKIP: test target mailing list: Series merged, no reason to check other mailing lists (test_mbox.TestMbox.test_target_mailing_list) --- Please address the issues identified and submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!
On Mon, 2024-09-02 at 00:39 -0700, Hemraj, Deepthi via lists.openembedded.org wrote: > From: Deepthi Hemraj <Deepthi.Hemraj@windriver.com> > > [YOCTO #15338] > Enabled ptest integration in gcc-sanitizers.inc to facilitate the proper testing and execution of address sanitizer (ASan) > > Defined the do_check task to execute ASan tests with proper environment setup and updated do_install_ptest_base to copy ASan test executables to the ptest directory. Included the run-ptest script to automate test execution. > > The run-ptest script iterates through executable files copied from the host, runs each on the target, and logs the outcomes as either 'PASS' or 'XFAIL'. XFAIL is because these tests are expected to fail on the host system as per dejagnu as { dg-shouldfail "asan" } > > Signed-off-by: Deepthi Hemraj <Deepthi.Hemraj@windriver.com> > --- > .../distro/include/ptest-packagelists.inc | 1 + > meta/recipes-devtools/gcc/gcc-14.2.inc | 2 + > meta/recipes-devtools/gcc/gcc-sanitizers.inc | 43 ++++++++++++++++ > meta/recipes-devtools/gcc/gcc/asan-exp.patch | 51 +++++++++++++++++++ > meta/recipes-devtools/gcc/gcc/run-ptest | 28 ++++++++++ > 5 files changed, 125 insertions(+) > create mode 100644 meta/recipes-devtools/gcc/gcc/asan-exp.patch > create mode 100755 meta/recipes-devtools/gcc/gcc/run-ptest Would it make more sense to run this as part of the other gcc testsuites? It looks like there is a lot of duplication in these changes including a lot of ptest code. I'm also quite concerned at the buildpaths exceptions, build paths would be pointless in target packages so I'm not sure why those are present. Cheers, Richard
Hi Richard, Thanks for checking the patch and your comment "Would it make more sense to run this as part of the other gcc testsuites?" Definitely Yes. We can add the sanitizer tests as part of the existing gcc testsuites. However, the sanitizer test was intentionally kept separate to avoid unnecessary building and testing of sanitizers that are not required for most users. The current activity was taken up when checking the sanitizers support for all targets. This testsuite will just test the sanitizers support quickly within few minutes. Running the complete testsuite to check the sanitizers would require few hours. Hence, we had planned and implemented the ptest specifically for sanitizer support. However, we will add it as part of the complete testsuite if that seems most profitable for everyone. Please let us know your views on the same so that we can work towards the same.
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc index 7b8418654a..13b15b8647 100644 --- a/meta/conf/distro/include/ptest-packagelists.inc +++ b/meta/conf/distro/include/ptest-packagelists.inc @@ -85,6 +85,7 @@ PTESTS_FAST = "\ wayland \ xz \ zlib \ + gcc-sanitizers \ " PTESTS_FAST:append:libc-glibc = " glibc-y2038-tests" PTESTS_PROBLEMS:remove:libc-glibc = "glibc-y2038-tests" diff --git a/meta/recipes-devtools/gcc/gcc-14.2.inc b/meta/recipes-devtools/gcc/gcc-14.2.inc index f05484cfc0..72cd93f4bd 100644 --- a/meta/recipes-devtools/gcc/gcc-14.2.inc +++ b/meta/recipes-devtools/gcc/gcc-14.2.inc @@ -68,6 +68,8 @@ SRC_URI = "${BASEURI} \ file://0023-Fix-install-path-of-linux64.h.patch \ file://0024-Avoid-hardcoded-build-paths-into-ppc-libgcc.patch \ file://0025-gcc-testsuite-tweaks-for-mips-OE.patch \ + file://asan-exp.patch \ + file://run-ptest \ " S = "${TMPDIR}/work-shared/gcc-${PV}-${PR}/${SOURCEDIR}" diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc b/meta/recipes-devtools/gcc/gcc-sanitizers.inc index 524ebd4ba4..bb617cc9de 100644 --- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc +++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc @@ -119,3 +119,46 @@ FILES:libtsan-dev += "\ FILES:libtsan-staticdev += "${libdir}/libtsan.a" FILES:${PN} = "${libdir}/*.spec ${libdir}/${TARGET_SYS}/${BINV}/include/sanitizer/*.h" + +require gcc-testsuite.inc +inherit ptest + +# test only asan.exp related testcases +MAKE_CHECK_RUNTESTFLAGS ??= "${MAKE_CHECK_BOARDARGS} asan-ptest.exp" + +# specific host and target dependencies required for test suite running +do_check[depends] += "dejagnu-native:do_populate_sysroot expect-native:do_populate_sysroot" +do_check[depends] += "virtual/libc:do_populate_sysroot" +# only depend on qemu if targeting linux user execution +do_check[depends] += "${@'qemu-native:do_populate_sysroot' if "user" in d.getVar('TOOLCHAIN_TEST_TARGET') else ''}" +do_check[prefuncs] += "extend_recipe_sysroot" +do_check[prefuncs] += "check_prepare" +do_check[dirs] = "${WORKDIR}/dejagnu ${B}" +do_check() { + export DEJAGNU="${WORKDIR}/dejagnu/site.exp" + oe_runmake -i check-gcc RUNTESTFLAGS="${MAKE_CHECK_RUNTESTFLAGS}" +} +addtask check after do_compile before do_install_ptest_base + +do_install_ptest_base() { + + + ls ${B}/gcc/testsuite/gcc/*.exe > exe_files + tmp_file=$(cat exe_files) + + echo "${tmp_file}" + + install -d ${D}${PTEST_PATH}/tests/sanitizers-ptest/ + + for f in ${tmp_file} + do + cp -rf ${f} ${D}${PTEST_PATH}/tests/sanitizers-ptest/ + done + + install -d ${D}${PTEST_PATH} + cp ${S}/../sources-unpack/run-ptest ${D}${PTEST_PATH}/ +} + +INSANE_SKIP:${PN}-ptest += "ldflags buildpaths" +INSANE_SKIP:${PN}-dbg += "buildpaths" +FILES_${PN} += "${PTEST_PATH}/" diff --git a/meta/recipes-devtools/gcc/gcc/asan-exp.patch b/meta/recipes-devtools/gcc/gcc/asan-exp.patch new file mode 100644 index 0000000000..6ad8db56ef --- /dev/null +++ b/meta/recipes-devtools/gcc/gcc/asan-exp.patch @@ -0,0 +1,51 @@ +Upstream-Status: Pending [gcc specific] +--- +diff --git a/gcc/testsuite/gcc.dg/asan/asan-ptest.exp b/gcc/testsuite/gcc.dg/asan/asan-ptest.exp +new file mode 100644 +index 000000000..4f05d90d0 +--- /dev/null ++++ b/gcc/testsuite/gcc.dg/asan/asan-ptest.exp +@@ -0,0 +1,43 @@ ++# Copyright (C) 2012-2024 Free Software Foundation, Inc. ++# ++# This file is part of GCC. ++# ++# GCC is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 3, or (at your option) ++# any later version. ++# ++# GCC is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with GCC; see the file COPYING3. If not see ++# <http://www.gnu.org/licenses/>. ++ ++# GCC testsuite that uses the `dg.exp' driver. ++ ++# Load support procs. ++load_lib gcc-dg.exp ++load_lib asan-dg.exp ++ ++# Initialize `dg'. ++dg-init ++# libasan uses libstdc++ so make sure we provide paths for it. ++asan_init 1 ++ ++# Main loop. ++if [check_effective_target_fsanitize_address] { ++ foreach testcase [lsort [glob -nocomplain $srcdir/$subdir/*.c ]] { ++ dg-test -keep-output $testcase "" "" ++ } ++ ++ foreach testcase [lsort [glob -nocomplain $srcdir/c-c++-common/asan/*.c ]] { ++ dg-test -keep-output $testcase "" "" ++ } ++} ++ ++# All done. ++asan_finish ++dg-finish diff --git a/meta/recipes-devtools/gcc/gcc/run-ptest b/meta/recipes-devtools/gcc/gcc/run-ptest new file mode 100755 index 0000000000..b8dcd16e95 --- /dev/null +++ b/meta/recipes-devtools/gcc/gcc/run-ptest @@ -0,0 +1,28 @@ +#!/bin/sh + +# This script is used to run sanitizers test suites + +ptestdir=$(dirname "$(readlink -f "$0")") +export SRC_PATH=$ptestdir + +cd $ptestdir/ +files=$(ls tests/sanitizers-ptest/*.exe) +total_testcases=$(ls tests/sanitizers-ptest/*.exe | wc -l) +passed=0 +failed=0 + +for f in $files; do + $f >/dev/null 2>&1 + retcode=$? + if [ $retcode -eq 0 ]; then + echo "PASS" : $f + ((passed++)) + else + echo "XFAIL" : $f + ((failed++)) + fi +done + +echo "Total Testcases: $total_testcases" +echo "Passed: $passed" +echo "Xfailed: $failed"