mbox series

[v5,00/11] mesa: upgrade to 25.0.2

Message ID 20250402-mesa-25-v5-0-ab35f021a83b@oss.qualcomm.com
Headers show
Series mesa: upgrade to 25.0.2 | expand

Message

Dmitry Baryshkov April 2, 2025, 3:18 p.m. UTC 
libclc is required in order to build Intel drivers in Mesa 24.1 and
later. Thus OE-Core is currently locked to the 24.0.x branch, which is
no longer maintained, having last release in June 2024.

A clean solution is to package Clang, then use it to build libclc and
SPIRV-LLVM-Translator (required for Mesa OpenCL support). However
creating proper recipes for Clang is a long process (ongoing by Khem
Raj).

This patchset attempts to solve the issue in a simpler way: by building
libclc and SPIRV-LLVM-Translator together with the LLVM. This way we
don't have to create target Clang recipes, there is no extra toolchain
to support, etc. Clang binaries are creted as a part of the build
process (in order to support building target packages), however it only
hits the LLVM sysroot-native and is only used by the LLVM target build.

Future work by Khem (once Clang is properly supported in OE-Core) should
allow us to create proper recipes for libclc and spirv-llvm-translator
and drop them from the llvm / clang recipe.

I think this approach provides a good balance between OE-Core being
stuck with the outdated Mesa release and being able to upgrade it,
bringing support for new hardware.

Last two patches of the series bring in RustiCL support, a new OpenCL
implementation within Mesa. They are optional for the sake of the
upgrade and can be dropped if there is any issue with those. RustiCL
support is mostly compile-tested at this point.

Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
---
Changes in v5:
- Extended runqemu script to also set GBM_BACKENDS_PATH
- Fixed LDFLAGS leak from native llvm-config --ldflags (which caused
  mesa-clc issues)
- Disabled BLAKE3 SIMD optimizations on x32, fixing the crash
- Disabled swarst Vulkan driver and llvmpipe Gallium driver on x32,
  fixing the crash
- Changed bindgen patch to pass -target to all bindgen invocations
- Link to v4: https://lore.kernel.org/r/20250331-mesa-25-v4-0-e62fc54a6efc@oss.qualcomm.com

Changes in v4:
- Fixed QA issues reported against lib32-mesa-megadriver (Richard)
- Removed release notes link from the mesa's commit message (Alexander)
- Added link to the rust.bindgen() issue to RustiCL-related
  cross-compilation patch (Alexander)
- Renamed 'spirv' PACKAGECONFIG option to 'spirv-llvm-translator' in
  order to be more precise
- Link to v3: https://lore.kernel.org/r/20250329-mesa-25-v3-0-f4d0328eb867@oss.qualcomm.com

Changes in v3:
- Dropped llvm/fix-native-compilation.patch and
  llvm/0001-cmake-modules-CrossCompile-pass-SPIR-V-headers-path-.patch
  in favour of installing prepare_builtins into the sysroot, dropped
  corresponding EXTRA_OECMAKE params
- Dropepd -DCLANG_TABLEGEN from llvm's EXTRA_OECMAKE
- Enabled llvm-config wrapper for nativesdk too, fixing QA errors for
  nativesdk-mesa
- Added --libdir handling to llvm-config, dropping the need for
  mesa/0001-meson-use-target-path-for-Clang-libdir.patch
- Dropped extra LIBCEXTENSION from BINDGEN_TARGET, it is already a part
  of TARGET_OS
- Expanded commit messages to explain the patches
- Link to v2: https://lore.kernel.org/r/20250327221807.2551544-1-dmitry.baryshkov@oss.qualcomm.com

Changes in v2:
- Dropped YOCTO_ALTERNATE_LIBDIR patch
- Reworked mesa / LLVM integration to always use get_option('libdir')
- Reenabled AMD Vulkan driver (got disabled because of c&p error)
- Moved mesa-clc options to the libclc PACKAGECONFIG to fix several
  build errors
- Link to v1: https://lore.kernel.org/r/20250327135520.3920189-1-dmitry.baryshkov@oss.qualcomm.com/

---
Dmitry Baryshkov (10):
      runqemu: also set GBM_BACKENDS_PATH
      llvm: move libLLVM.so.N.M to llvm-libllvm package
      llvm: use OECMAKE_SOURCEPATH to specify cmake dir
      llvm: handle libdir in llvm-config
      llvm: also use llvm-config wrapper for nativesdk recipes
      llvm: remove LLVM_LDFLAGS from llvm-config --ldflags output
      llvm: support building libclc
      llvm: add SPIRV-LLVM-Translator support
      mesa: upgrade 24.0.7 -> 25.0.2
      bindgen-cli: a tool to generate Rust bindings

Zoltán Böszörményi (1):
      mesa: add support for RustiCL under PACKAGECONFIG "opencl"

 meta/conf/distro/include/maintainers.inc           |   1 +
 .../bindgen-cli/bindgen-cli-crates.inc             | 186 +++++++++++++++++++++
 .../bindgen-cli/bindgen-cli_0.71.1.bb              |  19 +++
 ...g-remove-LLVM_LDFLAGS-from-ldflags-output.patch |  34 ++++
 meta/recipes-devtools/llvm/llvm/llvm-config        |   3 +
 .../llvm/llvm/spirv-internal-build.patch           |  30 ++++
 meta/recipes-devtools/llvm/llvm_20.1.0.bb          |  79 ++++++++-
 ...evert-meson-do-not-pull-in-clc-for-clover.patch |  53 ------
 ...d-Include-missing-llvm-IR-header-Module.h.patch |  41 -----
 .../files/0001-drisw-fix-build-without-dri3.patch  |  58 -------
 ...eno-don-t-encode-build-path-into-binaries.patch |  46 ++++-
 ...able-SIMD-blake-optimisations-on-x32-host.patch |  28 ++++
 ...vide-cross-compilation-arguments-to-LLVM-.patch |  67 ++++++++
 ...ext-don-t-try-zink-if-not-enabled-in-mesa.patch |  42 -----
 meta/recipes-graphics/mesa/mesa.inc                | 110 +++++++-----
 scripts/runqemu                                    |  13 +-
 16 files changed, 551 insertions(+), 259 deletions(-)
---
base-commit: dc2ef2cc1183a14cd3a05d388a1d0485a1bc8d20
change-id: 20250328-mesa-25-6b78a0ce2c90

Best regards,

Comments

Richard Purdie April 3, 2025, 7:33 a.m. UTC | #1 
On Wed, 2025-04-02 at 18:18 +0300, Dmitry Baryshkov via lists.openembedded.org wrote:
> libclc is required in order to build Intel drivers in Mesa 24.1 and
> later. Thus OE-Core is currently locked to the 24.0.x branch, which is
> no longer maintained, having last release in June 2024.
> 
> A clean solution is to package Clang, then use it to build libclc and
> SPIRV-LLVM-Translator (required for Mesa OpenCL support). However
> creating proper recipes for Clang is a long process (ongoing by Khem
> Raj).
> 
> This patchset attempts to solve the issue in a simpler way: by building
> libclc and SPIRV-LLVM-Translator together with the LLVM. This way we
> don't have to create target Clang recipes, there is no extra toolchain
> to support, etc. Clang binaries are creted as a part of the build
> process (in order to support building target packages), however it only
> hits the LLVM sysroot-native and is only used by the LLVM target build.
> 
> Future work by Khem (once Clang is properly supported in OE-Core) should
> allow us to create proper recipes for libclc and spirv-llvm-translator
> and drop them from the llvm / clang recipe.
> 
> I think this approach provides a good balance between OE-Core being
> stuck with the outdated Mesa release and being able to upgrade it,
> bringing support for new hardware.
> 
> Last two patches of the series bring in RustiCL support, a new OpenCL
> implementation within Mesa. They are optional for the sake of the
> upgrade and can be dropped if there is any issue with those. RustiCL
> support is mostly compile-tested at this point.
> 
> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
> ---
> Changes in v5:
> - Extended runqemu script to also set GBM_BACKENDS_PATH
> - Fixed LDFLAGS leak from native llvm-config --ldflags (which caused
>   mesa-clc issues)
> - Disabled BLAKE3 SIMD optimizations on x32, fixing the crash
> - Disabled swarst Vulkan driver and llvmpipe Gallium driver on x32,
>   fixing the crash
> - Changed bindgen patch to pass -target to all bindgen invocations
> - Link to v4: https://lore.kernel.org/r/20250331-mesa-25-v4-0-e62fc54a6efc@oss.qualcomm.com
> 
> Changes in v4:
> - Fixed QA issues reported against lib32-mesa-megadriver (Richard)
> - Removed release notes link from the mesa's commit message (Alexander)
> - Added link to the rust.bindgen() issue to RustiCL-related
>   cross-compilation patch (Alexander)
> - Renamed 'spirv' PACKAGECONFIG option to 'spirv-llvm-translator' in
>   order to be more precise
> - Link to v3: https://lore.kernel.org/r/20250329-mesa-25-v3-0-f4d0328eb867@oss.qualcomm.com
> 
> Changes in v3:
> - Dropped llvm/fix-native-compilation.patch and
>   llvm/0001-cmake-modules-CrossCompile-pass-SPIR-V-headers-path-.patch
>   in favour of installing prepare_builtins into the sysroot, dropped
>   corresponding EXTRA_OECMAKE params
> - Dropepd -DCLANG_TABLEGEN from llvm's EXTRA_OECMAKE
> - Enabled llvm-config wrapper for nativesdk too, fixing QA errors for
>   nativesdk-mesa
> - Added --libdir handling to llvm-config, dropping the need for
>   mesa/0001-meson-use-target-path-for-Clang-libdir.patch
> - Dropped extra LIBCEXTENSION from BINDGEN_TARGET, it is already a part
>   of TARGET_OS
> - Expanded commit messages to explain the patches
> - Link to v2: https://lore.kernel.org/r/20250327221807.2551544-1-dmitry.baryshkov@oss.qualcomm.com
> 
> Changes in v2:
> - Dropped YOCTO_ALTERNATE_LIBDIR patch
> - Reworked mesa / LLVM integration to always use get_option('libdir')
> - Reenabled AMD Vulkan driver (got disabled because of c&p error)
> - Moved mesa-clc options to the libclc PACKAGECONFIG to fix several
>   build errors
> - Link to v1: https://lore.kernel.org/r/20250327135520.3920189-1-dmitry.baryshkov@oss.qualcomm.com/
> 
> ---
> Dmitry Baryshkov (10):
>       runqemu: also set GBM_BACKENDS_PATH
>       llvm: move libLLVM.so.N.M to llvm-libllvm package
>       llvm: use OECMAKE_SOURCEPATH to specify cmake dir
>       llvm: handle libdir in llvm-config
>       llvm: also use llvm-config wrapper for nativesdk recipes
>       llvm: remove LLVM_LDFLAGS from llvm-config --ldflags output
>       llvm: support building libclc
>       llvm: add SPIRV-LLVM-Translator support
>       mesa: upgrade 24.0.7 -> 25.0.2
>       bindgen-cli: a tool to generate Rust bindings
> 
> Zoltán Böszörményi (1):
>       mesa: add support for RustiCL under PACKAGECONFIG "opencl"
> 
>  meta/conf/distro/include/maintainers.inc           |   1 +
>  .../bindgen-cli/bindgen-cli-crates.inc             | 186 +++++++++++++++++++++
>  .../bindgen-cli/bindgen-cli_0.71.1.bb              |  19 +++
>  ...g-remove-LLVM_LDFLAGS-from-ldflags-output.patch |  34 ++++
>  meta/recipes-devtools/llvm/llvm/llvm-config        |   3 +
>  .../llvm/llvm/spirv-internal-build.patch           |  30 ++++
>  meta/recipes-devtools/llvm/llvm_20.1.0.bb          |  79 ++++++++-
>  ...evert-meson-do-not-pull-in-clc-for-clover.patch |  53 ------
>  ...d-Include-missing-llvm-IR-header-Module.h.patch |  41 -----
>  .../files/0001-drisw-fix-build-without-dri3.patch  |  58 -------
>  ...eno-don-t-encode-build-path-into-binaries.patch |  46 ++++-
>  ...able-SIMD-blake-optimisations-on-x32-host.patch |  28 ++++
>  ...vide-cross-compilation-arguments-to-LLVM-.patch |  67 ++++++++
>  ...ext-don-t-try-zink-if-not-enabled-in-mesa.patch |  42 -----
>  meta/recipes-graphics/mesa/mesa.inc                | 110 +++++++-----
>  scripts/runqemu                                    |  13 +-
>  16 files changed, 551 insertions(+), 259 deletions(-)

I ran through testing with this and it is definitely improving. The
most problematic issue seen was this:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1374

which I suspect is uninative being bypassed. I have memories of dealing
with something to do with __isoc23_strtol but I can't remember the
details unfortunately.

meta-clang also failed:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/18/builds/667

which may or may not be related to khem's libdir feedback.

Cheers,

Richard