mbox series

[v3,00/10] clang-merge: Bring clang recipes from meta-clang

Message ID 20250425-clang-merge-v3-0-feea6f07d130@gmail.com
Headers show
Series clang-merge: Bring clang recipes from meta-clang | expand

Message

Khem Raj April 26, 2025, 2:35 a.m. UTC
Latest mesa 25+ needs libclang during build for tools like mesa_clc,
we enahnced already existing llvm recipe to build pieces of clang as
well, which worked for mesa's needs but broke static clang compiler
provided by meta-clang due to newfound conflicts, since clang is now
built with llvm recipe from core and also by meta-clang and they
compete for same namespace, an attempt to make them co-habit did reveal
more difficulties. Clang compiler from meta-clang is not able to use
llvm libraries from core since they patch them differently.

This patch series therefore brings clang and all recipes that build from
llvm sources into core. With this changeset, clang recipes can be removed
from meta-clang there is a pull request for that already [1]
As an aside, we only build clang once and use it for meta-clang needs
and core needs.

With this, meta-clang can depend upon core layer to provide clang and
related recipes, meta-clang will still provide the toolchain policies
around clang based toolchains. Clang compiler from core, while usable
for mesa is not yet fully usable out of box, since we need to workout
the multi-toolchain architecture for core, which will be done in next
phase after this merge.

[1] https://github.com/kraj/meta-clang/pull/1088

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
Changes in v3:
- Fix building mesa with openCL packageconfig.
- Fix buildpaths in libMesaOpenCL.so.1.0.0.
- Add mesa RPROVIDES for libclc and spirv-llvm-translator
- Link to v2: https://lore.kernel.org/r/20250424-clang-merge-v2-0-0a51d0c1940d@gmail.com

Changes in v2:
- Address Feedback from Alex and Dmitry.
- Squash similar commits to reduce number of patches.
- Move native toolchain file generation to cmake bbclass instead of
  adding new class
- Fix space/tabs indentation inconsistencies
- Reorder the patch sequence to avoid errors in future git bisects
- Fix use before define string problem in one of the patches
- Link to v1: https://lore.kernel.org/r/20250424-clang-merge-v1-0-5a492a8461aa@gmail.com

---
Khem Raj (10):
      cmake.bbclass: Enhance to emit a native toolchain cmake file
      toolchain/clang: Remove duplicate RANLIB setting
      sstatesig: Handle special case of llvm-project-source shared-workdir
      multilib.conf: Add llvm-project-source recipe to NON_MULTILIB_RECIPES
      kernel-arch.bbclass: Do not use weak assignment for TOOLCHAIN
      clang: Merge llvm/clang family recipes from meta-clang
      clang: RPROVIDE spirv-llvm-translator and libclc
      mesa: Pass needed clang flags via BINDGEN_EXTRA_CLANG_ARGS
      mesa: Fix header search paths
      maintainers.inc: Add myself as maintainer for clang family of recipes

 meta/classes-recipe/cmake.bbclass                  |  58 +++
 meta/classes-recipe/kernel-arch.bbclass            |   2 +-
 meta/conf/distro/include/maintainers.inc           |  11 +-
 meta/conf/multilib.conf                            |   2 +-
 meta/conf/toolchain/clang.inc                      |   1 -
 meta/lib/oe/sstatesig.py                           |   3 +
 .../clang/clang-cross-canadian_git.bb              |  36 ++
 meta/recipes-devtools/clang/clang-cross_git.bb     |  39 ++
 meta/recipes-devtools/clang/clang-crosssdk_git.bb  |  33 ++
 ...ind-libunwind-headers-when-LIBCXXABI_LIBU.patch |  60 +++
 ...er-rt-support-a-new-embedded-linux-target.patch | 309 +++++++++++++
 ...-Simplify-cross-compilation.-Don-t-use-na.patch |  44 ++
 ...LibraryInfo-Undefine-libc-functions-if-th.patch |  90 ++++
 ...allow-env-override-of-exe-and-libdir-path.patch |  71 +++
 ...-clang-driver-Check-sysroot-for-ldso-path.patch |  78 ++++
 ...iver-tools.cpp-Add-lssp_nonshared-on-musl.patch |  32 ++
 .../0008-clang-Prepend-trailing-to-sysroot.patch   |  39 ++
 ...inside-the-target-sysroot-for-compiler-ru.patch |  41 ++
 ...ang-Define-releative-gcc-installation-dir.patch | 123 +++++
 ...pthread-and-ldl-along-with-lunwind-for-st.patch |  35 ++
 ..._EXECUTABLE-when-cross-compiling-for-nati.patch |  24 +
 .../0013-Check-for-atomic-double-intrinsics.patch  |  34 ++
 ...configure-for-packages-using-find_package.patch |   2 +-
 ...esource-dir-location-for-cross-toolchains.patch |  50 +++
 ...r-Add-dyld-prefix-when-checking-sysroot-f.patch |  79 ++++
 .../0017-clang-Use-python3-in-python-scripts.patch |  35 ++
 ...Insert-anchor-for-adding-OE-distro-vendor.patch | 167 +++++++
 ...-Do-not-use-backtrace-APIs-on-non-glibc-l.patch |  68 +++
 ...86-triple-for-non-debian-multiarch-linux-.patch |  28 ++
 ...0021-libunwind-Added-unw_backtrace-method.patch |  56 +++
 .../0022-lldb-Link-with-libatomic-on-x86.patch     |  33 ++
 ...023-compiler-rt-Enable-__int128-for-ppc32.patch |  73 +++
 ...-Do-not-use-cmake-infra-to-detect-libzstd.patch |  62 +++
 ...ler-rt-Fix-stat-struct-s-size-for-O32-ABI.patch |  46 ++
 ...-Undef-_TIME_BITS-along-with-_FILE_OFFSET.patch |  43 ++
 ...s-Gnu.cpp-ARMLibDirs-search-also-in-lib32.patch |  81 ++++
 ...vm-Add-OE-specific-ABI-triple-for-N32-ABI.patch |  78 ++++
 ...d-libunwind.pc.in-and-llvm-config-scripts.patch |  90 ++++
 ...py-respect-LLVM_LIBDIR_SUFFIX-like-other-.patch |  92 ++++
 ...r-rt-Do-not-pass-target-to-clang-compiler.patch |  29 ++
 .../clang/clang/0032-Fix-build-on-ppc64-musl.patch |  97 ++++
 ...d-a-build-option-to-disable-building-dexp.patch |  85 ++++
 ...itter-sort-ClassInfo-lists-by-name-as-we.patch} |   4 +-
 ...-remove-LLVM_LDFLAGS-from-ldflags-output.patch} |  12 +-
 ...ot-emit-date-and-time-into-generate-files.patch |  37 ++
 .../clang}/spirv-internal-build.patch              |   0
 .../clang}/spirv-shared-library.patch              |   0
 meta/recipes-devtools/clang/clang_git.bb           | 499 +++++++++++++++++++++
 meta/recipes-devtools/clang/common-clang.inc       |  24 +
 meta/recipes-devtools/clang/common-source.inc      |  17 +
 meta/recipes-devtools/clang/common.inc             |  83 ++++
 .../clang/compiler-rt-sanitizers_git.bb            | 139 ++++++
 meta/recipes-devtools/clang/compiler-rt_git.bb     | 125 ++++++
 meta/recipes-devtools/clang/libcxx_git.bb          | 119 +++++
 meta/recipes-devtools/clang/llvm-project-source.bb |  12 +
 .../recipes-devtools/clang/llvm-project-source.inc |  99 ++++
 .../recipes-devtools/clang/nativesdk-clang-glue.bb |  36 ++
 meta/recipes-devtools/clang/openmp_git.bb          |  67 +++
 .../0007-llvm-allow-env-override-of-exe-path.patch |  36 --
 meta/recipes-devtools/llvm/llvm/llvm-config        |  54 ---
 meta/recipes-devtools/llvm/llvm_20.1.2.bb          | 233 ----------
 .../0001-clover-Don-t-include-libclc-headers.patch | 143 ++++++
 ...ver-Do-not-use-LLVM_LIBRARY_DIR-for-FALLB.patch |  34 ++
 meta/recipes-graphics/mesa/mesa.inc                |   3 +
 64 files changed, 3898 insertions(+), 337 deletions(-)
---
base-commit: 4726c180470222b343adffa3c2eda118d688f5fc
change-id: 20250423-clang-merge-ca9130ae95af

Best regards,

Comments

Dmitry Baryshkov April 26, 2025, 3:54 a.m. UTC | #1
On Fri, Apr 25, 2025 at 07:35:02PM -0700, Khem Raj wrote:
> Latest mesa 25+ needs libclang during build for tools like mesa_clc,
> we enahnced already existing llvm recipe to build pieces of clang as
> well, which worked for mesa's needs but broke static clang compiler
> provided by meta-clang due to newfound conflicts, since clang is now
> built with llvm recipe from core and also by meta-clang and they
> compete for same namespace, an attempt to make them co-habit did reveal
> more difficulties. Clang compiler from meta-clang is not able to use
> llvm libraries from core since they patch them differently.
> 
> This patch series therefore brings clang and all recipes that build from
> llvm sources into core. With this changeset, clang recipes can be removed
> from meta-clang there is a pull request for that already [1]
> As an aside, we only build clang once and use it for meta-clang needs
> and core needs.
> 
> With this, meta-clang can depend upon core layer to provide clang and
> related recipes, meta-clang will still provide the toolchain policies
> around clang based toolchains. Clang compiler from core, while usable
> for mesa is not yet fully usable out of box, since we need to workout
> the multi-toolchain architecture for core, which will be done in next
> phase after this merge.

After testing several configurations, only nativesdk-mesa with enabled OpenCL fails at building rusticl_llvm_bindings.rs:

/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/cstdlib:75:15: fatal error: 'stdlib.h' file not found
Unable to generate bindings: clang diagnosed error: /../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/cstdlib:75:15: fatal error: 'stdlib.h' file not found

All other configuraions seems to work. I think it can be merged now with
nativesdk-mesa with OpenCL being fixed later.

> 
> [1] https://github.com/kraj/meta-clang/pull/1088
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
> Changes in v3:
> - Fix building mesa with openCL packageconfig.
> - Fix buildpaths in libMesaOpenCL.so.1.0.0.
> - Add mesa RPROVIDES for libclc and spirv-llvm-translator
> - Link to v2: https://lore.kernel.org/r/20250424-clang-merge-v2-0-0a51d0c1940d@gmail.com
> 
> Changes in v2:
> - Address Feedback from Alex and Dmitry.
> - Squash similar commits to reduce number of patches.
> - Move native toolchain file generation to cmake bbclass instead of
>   adding new class
> - Fix space/tabs indentation inconsistencies
> - Reorder the patch sequence to avoid errors in future git bisects
> - Fix use before define string problem in one of the patches
> - Link to v1: https://lore.kernel.org/r/20250424-clang-merge-v1-0-5a492a8461aa@gmail.com
>