diff mbox series

[RFC,DO,NOT,MERGE] insane: check for 'unsafe for cross-compilation' in log.do_compile and log.do_install again

Message ID 20220806090714.2001241-1-Martin.Jansa@gmail.com
State New
Headers show
Series [RFC,DO,NOT,MERGE] insane: check for 'unsafe for cross-compilation' in log.do_compile and log.do_install again | expand

Commit Message

Martin Jansa Aug. 6, 2022, 9:07 a.m. UTC 
* only a RFC to discuss what might prevent
  --enable-poison-system-directories=true in gcc-cross to work or to
  test this on AB to see if there are other places where these issues
  sneaked in without being detected

* it was removed in:
  https://git.openembedded.org/openembedded-core/commit/?id=a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1
  but looks like the "follow-up patches":
    d039d6fbfc gcc-cross: make use of the system include directories fatal
    1fcaa5f452 gcc: add an option for --enable-poison-system-directories to be fatal
  don't work anymore? I was checking if e.g. -Wno-error is used in
  webkitgtk to disable them, but it's not there, so something else went
  wrong, because in webkit-gtk log.do_compile I've just noticed:

  # grep unsafe poky/build/tmp/work/core2-64-poky-linux/webkitgtk/2.36.4-r0/temp/log.do_compile.3901410 | sort | uniq -c
      6 cc1: warning: include location "/usr/include/gstreamer-1.0" is unsafe for cross-compilation [-Wpoison-system-directories]
   1267 cc1plus: warning: include location "/usr/include/gstreamer-1.0"	is unsafe for cross-compilation [-Wpoison-system-directories]

* and with this it confirms the same in webkitgtk:

  ERROR: webkitgtk-2.36.4-r0 do_compile: QA Issue: This /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/webkitgtk/2.36.4-r0/temp/log.do_compile log indicates 'unsafe for cross-compilation' or '-Wpoison-system-directories' errors, it looked at host include and/or library paths [compile-unsafe]

  and this was the only error (when building webkitgtk from scratch)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/classes/insane.bbclass | 38 ++++++++++++++++++++++++++++++++-----
 1 file changed, 33 insertions(+), 5 deletions(-)

Comments

Jose Quaresma Aug. 6, 2022, 11:11 a.m. UTC | #1 
Hi Martin,

Martin Jansa <Martin.Jansa@gmail.com> escreveu no dia sábado, 6/08/2022
à(s) 10:07:

> * only a RFC to discuss what might prevent
>   --enable-poison-system-directories=true in gcc-cross to work or to
>   test this on AB to see if there are other places where these issues
>   sneaked in without being detected
>
> * it was removed in:
>
> https://git.openembedded.org/openembedded-core/commit/?id=a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1
>   but looks like the "follow-up patches":
>     d039d6fbfc gcc-cross: make use of the system include directories fatal
>     1fcaa5f452 gcc: add an option for --enable-poison-system-directories
> to be fatal
>   don't work anymore? I was checking if e.g. -Wno-error is used in
>   webkitgtk to disable them, but it's not there, so something else went
>   wrong, because in webkit-gtk log.do_compile I've just noticed:
>
>   # grep unsafe
> poky/build/tmp/work/core2-64-poky-linux/webkitgtk/2.36.4-r0/temp/log.do_compile.3901410
> | sort | uniq -c
>       6 cc1: warning: include location "/usr/include/gstreamer-1.0" is
> unsafe for cross-compilation [-Wpoison-system-directories]
>    1267 cc1plus: warning: include location "/usr/include/gstreamer-1.0" is
> unsafe for cross-compilation [-Wpoison-system-directories]
>
> * and with this it confirms the same in webkitgtk:
>
>   ERROR: webkitgtk-2.36.4-r0 do_compile: QA Issue: This
> /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/webkitgtk/2.36.4-r0/temp/log.do_compile
> log indicates 'unsafe for cross-compilation' or
> '-Wpoison-system-directories' errors, it looked at host include and/or
> library paths [compile-unsafe]
>
>   and this was the only error (when building webkitgtk from scratch)
>
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>  meta/classes/insane.bbclass | 38 ++++++++++++++++++++++++++++++++-----
>  1 file changed, 33 insertions(+), 5 deletions(-)
>
> diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> index c8b434bb54..9ce5ae0f6f 100644
> --- a/meta/classes/insane.bbclass
> +++ b/meta/classes/insane.bbclass
> @@ -35,8 +35,8 @@ ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch
> pkgconfig la \
>              version-going-backwards expanded-d invalid-chars \
>              license-checksum dev-elf file-rdeps configure-unsafe \
>              configure-gettext perllocalpod shebang-size \
> -            already-stripped installed-vs-shipped ldflags
> compile-host-path \
> -            install-host-path pn-overrides unknown-configure-option \
> +            already-stripped installed-vs-shipped ldflags compile-unsafe \
> +            install-unsafe pn-overrides unknown-configure-option \
>              useless-rpaths rpaths staticdev empty-dirs \
>              "
>  # Add usrmerge QA check based on distro feature
> @@ -1231,6 +1231,31 @@ python do_qa_patch() {
>                     bb.error("Missing Upstream-Status in patch\n%s\nPlease
> add according to %s ." % (fullpath, guidelines))
>  }
>
> +def check_poisoned_system_directories(file):
> +    import subprocess
> +
> +    statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" %
> file
> +    if subprocess.call(statement, shell=True) == 0:
> +         return """This %s log indicates 'unsafe for cross-compilation'
> or '-Wpoison-system-directories' errors, it looked at host include and/or
> library paths""" % file
> +
> +python do_qa_compile() {
> +    log = os.path.join(d.getVar('T'),"log.do_compile")
> +    skip = (d.getVar('INSANE_SKIP') or "").split()
> +    if 'compile-unsafe' not in skip and os.path.exists(log):
> +        error_msg = check_poisoned_system_directories(log)
> +        if error_msg:
> +            oe.qa.handle_error("compile-unsafe", error_msg, d)
> +}
> +
> +python do_qa_install() {
> +    log = os.path.join(d.getVar('T'),"log.do_install")
> +    skip = (d.getVar('INSANE_SKIP') or "").split()
> +    if 'install-unsafe' not in skip and os.path.exists(log):
> +        error_msg = check_poisoned_system_directories(log)
> +        if error_msg:
> +            oe.qa.handle_error("install-unsafe", error_msg, d)
> +}
> +
>  python do_qa_configure() {
>      import subprocess
>
> @@ -1253,9 +1278,8 @@ python do_qa_configure() {
>              statement = "grep -q -F -e 'is unsafe for cross-compilation'
> %s" % \
>                          os.path.join(root,"config.log")
>              if "config.log" in files:
> -                if subprocess.call(statement, shell=True) == 0:
> -                    error_msg = """This autoconf log indicates errors, it
> looked at host include and/or library paths while determining system
> capabilities.
> -Rerun configure task after fixing this."""
> +                error_msg =
> check_poisoned_system_directories(os.path.join(root,"config.log"))
> +                if error_msg:
>                      oe.qa.handle_error("configure-unsafe", error_msg, d)
>
>              if "configure.ac" in files:
> @@ -1356,6 +1380,10 @@ do_patch[postfuncs] += "do_qa_patch "
>  #addtask qa_configure after do_configure before do_compile
>  do_configure[postfuncs] += "do_qa_configure "
>
> +# Check poison-system-directories in log.do_compile and log.do_install
> +do_compile[postfuncs] += "do_qa_compile"
> +do_compile[postfuncs] += "do_qa_install"
>

I think this is a typo and the second do_compile is do_install

Jose

+
>  # Check does S exist.
>  do_unpack[postfuncs] += "do_qa_unpack"
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#168969):
> https://lists.openembedded.org/g/openembedded-core/message/168969
> Mute This Topic: https://lists.openembedded.org/mt/92851436/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Martin Jansa Sept. 1, 2022, 5:53 p.m. UTC | #2 
FWIW: boost can also trigger this issue now:

    "x86_64-webos-linux-g++" "-m64" "-march=core2" "-mtune=core2" "-msse3"
"-mfpmath=sse" "-Wl,-O1" "-Wl,--hash-style=gnu" "-Wl,--as-needed"
"-fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0=/usr/src/debug/boost/1.80.0-r0"
"-fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0=/usr/src/debug/boost/1.80.0-r0"
"-fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build=/usr/src/debug/boost/1.80.0-r0"
"-fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build=/usr/src/debug/boost/1.80.0-r0"
"-fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot="
"-fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot="
"-fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot-native="
"--sysroot=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot"
  -fvisibility-inlines-hidden  -O2 -pipe -g -feliminate-unused-debug-types
-fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0=/usr/src/debug/boost/1.80.0-r0
 -fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0=/usr/src/debug/boost/1.80.0-r0
 -fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build=/usr/src/debug/boost/1.80.0-r0
 -fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build=/usr/src/debug/boost/1.80.0-r0
 -fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot=
 -fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot=
 -fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot-native=
 -fvisibility-inlines-hidden -m64 -pthread -O3 -finline-functions
-Wno-inline -Wall -fvisibility=hidden -Wno-long-long
-Wno-deprecated-declarations -DBOOST_ALL_NO_LIB=1
-DBOOST_CHRONO_STATIC_LINK=1 -DBOOST_LOCALE_NO_WINAPI_BACKEND=1
-DBOOST_LOCALE_WITH_ICONV=1 -DBOOST_LOCALE_WITH_ICU=1
-DBOOST_THREAD_BUILD_LIB=1 -DBOOST_THREAD_NO_LIB=1 -DBOOST_THREAD_POSIX
-DBOOST_THREAD_USE_LIB=1 -DNDEBUG  -I"."
-I"/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0/libs/locale/src"
-I"/usr/include"  -c -o
"/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build/boost/bin.v2/libs/locale/build/f2e0de3f303a0b3f8e80823b216d5db4/posix/converter.o"
"/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0/libs/locale/src/boost/locale/posix/converter.cpp"

cc1plus: warning: include location "/usr/include" is unsafe for
cross-compilation [-Wpoison-system-directories]

On Sat, Aug 6, 2022 at 2:55 PM Martin Jansa via lists.openembedded.org
<Martin.Jansa=gmail.com@lists.openembedded.org> wrote:

> * it was removed in:
>
> https://git.openembedded.org/openembedded-core/commit/?id=a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1
>   but looks like the "follow-up patches":
>     d039d6fbfc gcc-cross: make use of the system include directories fatal
>     1fcaa5f452 gcc: add an option for --enable-poison-system-directories
> to be fatal
>   don't work anymore? I was checking if e.g. -Wno-error is used in
>   webkitgtk to disable them, but it's not there, so something else went
>   wrong, because in webkit-gtk log.do_compile I've just noticed:
>
>   # grep unsafe
> poky/build/tmp/work/core2-64-poky-linux/webkitgtk/2.36.4-r0/temp/log.do_compile.3901410
> | sort | uniq -c
>       6 cc1: warning: include location "/usr/include/gstreamer-1.0" is
> unsafe for cross-compilation [-Wpoison-system-directories]
>    1267 cc1plus: warning: include location "/usr/include/gstreamer-1.0" is
> unsafe for cross-compilation [-Wpoison-system-directories]
>
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>
>  v2: fixed copy&paste issue in postfuncs reported by Jose
>
>  meta/classes/insane.bbclass | 38 ++++++++++++++++++++++++++++++++-----
>  1 file changed, 33 insertions(+), 5 deletions(-)
>
> diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> index c8b434bb54..2cb730111b 100644
> --- a/meta/classes/insane.bbclass
> +++ b/meta/classes/insane.bbclass
> @@ -35,8 +35,8 @@ ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch
> pkgconfig la \
>              version-going-backwards expanded-d invalid-chars \
>              license-checksum dev-elf file-rdeps configure-unsafe \
>              configure-gettext perllocalpod shebang-size \
> -            already-stripped installed-vs-shipped ldflags
> compile-host-path \
> -            install-host-path pn-overrides unknown-configure-option \
> +            already-stripped installed-vs-shipped ldflags compile-unsafe \
> +            install-unsafe pn-overrides unknown-configure-option \
>              useless-rpaths rpaths staticdev empty-dirs \
>              "
>  # Add usrmerge QA check based on distro feature
> @@ -1231,6 +1231,31 @@ python do_qa_patch() {
>                     bb.error("Missing Upstream-Status in patch\n%s\nPlease
> add according to %s ." % (fullpath, guidelines))
>  }
>
> +def check_poisoned_system_directories(file):
> +    import subprocess
> +
> +    statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" %
> file
> +    if subprocess.call(statement, shell=True) == 0:
> +         return """This %s log indicates 'unsafe for cross-compilation'
> or '-Wpoison-system-directories' errors, it looked at host include and/or
> library paths""" % file
> +
> +python do_qa_compile() {
> +    log = os.path.join(d.getVar('T'),"log.do_compile")
> +    skip = (d.getVar('INSANE_SKIP') or "").split()
> +    if 'compile-unsafe' not in skip and os.path.exists(log):
> +        error_msg = check_poisoned_system_directories(log)
> +        if error_msg:
> +            oe.qa.handle_error("compile-unsafe", error_msg, d)
> +}
> +
> +python do_qa_install() {
> +    log = os.path.join(d.getVar('T'),"log.do_install")
> +    skip = (d.getVar('INSANE_SKIP') or "").split()
> +    if 'install-unsafe' not in skip and os.path.exists(log):
> +        error_msg = check_poisoned_system_directories(log)
> +        if error_msg:
> +            oe.qa.handle_error("install-unsafe", error_msg, d)
> +}
> +
>  python do_qa_configure() {
>      import subprocess
>
> @@ -1253,9 +1278,8 @@ python do_qa_configure() {
>              statement = "grep -q -F -e 'is unsafe for cross-compilation'
> %s" % \
>                          os.path.join(root,"config.log")
>              if "config.log" in files:
> -                if subprocess.call(statement, shell=True) == 0:
> -                    error_msg = """This autoconf log indicates errors, it
> looked at host include and/or library paths while determining system
> capabilities.
> -Rerun configure task after fixing this."""
> +                error_msg =
> check_poisoned_system_directories(os.path.join(root,"config.log"))
> +                if error_msg:
>                      oe.qa.handle_error("configure-unsafe", error_msg, d)
>
>              if "configure.ac" in files:
> @@ -1356,6 +1380,10 @@ do_patch[postfuncs] += "do_qa_patch "
>  #addtask qa_configure after do_configure before do_compile
>  do_configure[postfuncs] += "do_qa_configure "
>
> +# Check poison-system-directories in log.do_compile and log.do_install
> +do_compile[postfuncs] += "do_qa_compile"
> +do_install[postfuncs] += "do_qa_install"
> +
>  # Check does S exist.
>  do_unpack[postfuncs] += "do_qa_unpack"
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#168973):
> https://lists.openembedded.org/g/openembedded-core/message/168973
> Mute This Topic: https://lists.openembedded.org/mt/92853487/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Ross Burton Sept. 6, 2022, 3:19 p.m. UTC | #3 
On 6 Aug 2022, at 10:07, Martin Jansa <martin.jansa@gmail.com> wrote:
> 
> * only a RFC to discuss what might prevent
>  --enable-poison-system-directories=true in gcc-cross to work or to
>  test this on AB to see if there are other places where these issues
>  sneaked in without being detected

This is… odd.

There’s a test case for this in oe-selftest, specifically the poison recipe which does a compile with -I/usr/include, and it fails:

cc1: error: include location "/usr/include" is unsafe for cross-compilation [-Werror=poison-system-directories]
cc1: some warnings being treated as errors
WARNING: exit code 1 from a shell command.
ERROR: Task (/home/ross/Yocto/poky/meta-selftest/recipes-test/poison/poison.bb:do_compile) failed with exit code ‘1'

Can you confirm that your gcc-cross is being passed --enable-poison-system-directories=error when configured?  =true means just warnings, you need =error.

Ross
diff mbox series

Patch

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index c8b434bb54..9ce5ae0f6f 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -35,8 +35,8 @@  ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
             version-going-backwards expanded-d invalid-chars \
             license-checksum dev-elf file-rdeps configure-unsafe \
             configure-gettext perllocalpod shebang-size \
-            already-stripped installed-vs-shipped ldflags compile-host-path \
-            install-host-path pn-overrides unknown-configure-option \
+            already-stripped installed-vs-shipped ldflags compile-unsafe \
+            install-unsafe pn-overrides unknown-configure-option \
             useless-rpaths rpaths staticdev empty-dirs \
             "
 # Add usrmerge QA check based on distro feature
@@ -1231,6 +1231,31 @@  python do_qa_patch() {
                    bb.error("Missing Upstream-Status in patch\n%s\nPlease add according to %s ." % (fullpath, guidelines))
 }
 
+def check_poisoned_system_directories(file):
+    import subprocess
+
+    statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" % file
+    if subprocess.call(statement, shell=True) == 0:
+         return """This %s log indicates 'unsafe for cross-compilation' or '-Wpoison-system-directories' errors, it looked at host include and/or library paths""" % file
+
+python do_qa_compile() {
+    log = os.path.join(d.getVar('T'),"log.do_compile")
+    skip = (d.getVar('INSANE_SKIP') or "").split()
+    if 'compile-unsafe' not in skip and os.path.exists(log):
+        error_msg = check_poisoned_system_directories(log)
+        if error_msg:
+            oe.qa.handle_error("compile-unsafe", error_msg, d)
+}
+
+python do_qa_install() {
+    log = os.path.join(d.getVar('T'),"log.do_install")
+    skip = (d.getVar('INSANE_SKIP') or "").split()
+    if 'install-unsafe' not in skip and os.path.exists(log):
+        error_msg = check_poisoned_system_directories(log)
+        if error_msg:
+            oe.qa.handle_error("install-unsafe", error_msg, d)
+}
+
 python do_qa_configure() {
     import subprocess
 
@@ -1253,9 +1278,8 @@  python do_qa_configure() {
             statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" % \
                         os.path.join(root,"config.log")
             if "config.log" in files:
-                if subprocess.call(statement, shell=True) == 0:
-                    error_msg = """This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities.
-Rerun configure task after fixing this."""
+                error_msg = check_poisoned_system_directories(os.path.join(root,"config.log"))
+                if error_msg:
                     oe.qa.handle_error("configure-unsafe", error_msg, d)
 
             if "configure.ac" in files:
@@ -1356,6 +1380,10 @@  do_patch[postfuncs] += "do_qa_patch "
 #addtask qa_configure after do_configure before do_compile
 do_configure[postfuncs] += "do_qa_configure "
 
+# Check poison-system-directories in log.do_compile and log.do_install
+do_compile[postfuncs] += "do_qa_compile"
+do_compile[postfuncs] += "do_qa_install"
+
 # Check does S exist.
 do_unpack[postfuncs] += "do_qa_unpack"