diff mbox series

libclc: split out of clang

Message ID 20250715143139.1829393-1-ross.burton@arm.com
State New
Headers show
Series libclc: split out of clang | expand

Commit Message

Ross Burton July 15, 2025, 2:31 p.m. UTC
Split the libclc subproject out of the clang recipe and into a dedicated
libclc recipe.

Verified that nothing is dropped by adding clang and libclc to an image
and verifying that the file list is the same before and after this
change.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/conf/distro/include/maintainers.inc  |  1 +
 meta/recipes-devtools/clang/clang_git.bb  | 17 +++----
 meta/recipes-devtools/clang/libclc_git.bb | 54 +++++++++++++++++++++++
 3 files changed, 61 insertions(+), 11 deletions(-)
 create mode 100644 meta/recipes-devtools/clang/libclc_git.bb

Comments

patchtest@automation.yoctoproject.org July 15, 2025, 2:45 p.m. UTC | #1
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/libclc-split-out-of-clang.patch

FAIL: test lic files chksum modified not mentioned: LIC_FILES_CHKSUM changed without "License-Update:" tag and description in commit message (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)

PASS: pretest src uri left files (test_metadata.TestMetadata.pretest_src_uri_left_files)
PASS: test CVE check ignore (test_metadata.TestMetadata.test_cve_check_ignore)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test src uri left files (test_metadata.TestMetadata.test_src_uri_left_files)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!
Dmitry Baryshkov July 15, 2025, 2:52 p.m. UTC | #2
On Tue, Jul 15, 2025 at 03:31:38PM +0100, Ross Burton wrote:
> Split the libclc subproject out of the clang recipe and into a dedicated
> libclc recipe.

I miss the 'why' part of the commit message. Otherwise LGTM.

> 
> Verified that nothing is dropped by adding clang and libclc to an image
> and verifying that the file list is the same before and after this
> change.
> 
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  meta/conf/distro/include/maintainers.inc  |  1 +
>  meta/recipes-devtools/clang/clang_git.bb  | 17 +++----
>  meta/recipes-devtools/clang/libclc_git.bb | 54 +++++++++++++++++++++++
>  3 files changed, 61 insertions(+), 11 deletions(-)
>  create mode 100644 meta/recipes-devtools/clang/libclc_git.bb
Ross Burton July 15, 2025, 3 p.m. UTC | #3
On 15 Jul 2025, at 15:52, Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> wrote:
> 
> On Tue, Jul 15, 2025 at 03:31:38PM +0100, Ross Burton wrote:
>> Split the libclc subproject out of the clang recipe and into a dedicated
>> libclc recipe.
> 
> I miss the 'why' part of the commit message. Otherwise LGTM.

Yes, good point.

Because building _all_ of clang when you don’t need it is painful, and introduces a large recipe build that bottlenecks everything else.

Adding clang and using it in mesa resulted in a massive increase in build time due to mesa and thus all of clang needed to be built twice.  I’m trying to recover some lost ground, by splitting out components. Even if they’re all used in the end, by not doing the build in a monolithic lump you get to run more tasks in parallel.

Ross
Alexander Kanavin July 15, 2025, 3:05 p.m. UTC | #4
On Tue, 15 Jul 2025 at 17:01, Ross Burton via lists.openembedded.org
<ross.burton=arm.com@lists.openembedded.org> wrote:
> Yes, good point.
>
> Because building _all_ of clang when you don’t need it is painful, and introduces a large recipe build that bottlenecks everything else.
>
> Adding clang and using it in mesa resulted in a massive increase in build time due to mesa and thus all of clang needed to be built twice.  I’m trying to recover some lost ground, by splitting out components. Even if they’re all used in the end, by not doing the build in a monolithic lump you get to run more tasks in parallel.

I'd also add that this has been mentioned in weekly status emails for
the past many many weeks. Those emails are a good way to know what are
the current pain points in the core project where help is appreciated
(and ideally joining the weekly tech meeting that starts right now).

Alex
Dmitry Baryshkov July 15, 2025, 3:53 p.m. UTC | #5
On 15/07/2025 18:05, Alexander Kanavin wrote:
> On Tue, 15 Jul 2025 at 17:01, Ross Burton via lists.openembedded.org
> <ross.burton=arm.com@lists.openembedded.org> wrote:
>> Yes, good point.
>>
>> Because building _all_ of clang when you don’t need it is painful, and introduces a large recipe build that bottlenecks everything else.
>>
>> Adding clang and using it in mesa resulted in a massive increase in build time due to mesa and thus all of clang needed to be built twice.  I’m trying to recover some lost ground, by splitting out components. Even if they’re all used in the end, by not doing the build in a monolithic lump you get to run more tasks in parallel.

There were patches to get rid of the second clang build.

Anyway, this should be a part of the commit message.

> 
> I'd also add that this has been mentioned in weekly status emails for
> the past many many weeks. Those emails are a good way to know what are
> the current pain points in the core project where help is appreciated
> (and ideally joining the weekly tech meeting that starts right now).
Ross Burton July 15, 2025, 4:29 p.m. UTC | #6
On 15 Jul 2025, at 16:53, Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> wrote:
> 
> There were patches to get rid of the second clang build.

That haven’t merged?  Can you point me at the discussion for these because I’m struggling to remember what ones they were.

Ross
Dmitry Baryshkov July 15, 2025, 5:18 p.m. UTC | #7
On Tue, 15 Jul 2025 at 19:29, Ross Burton <Ross.Burton@arm.com> wrote:
>
> On 15 Jul 2025, at 16:53, Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> wrote:
> >
> > There were patches to get rid of the second clang build.
>
> That haven’t merged?  Can you point me at the discussion for these because I’m struggling to remember what ones they were.

https://lore.kernel.org/openembedded-core/20250624-mesa-libclc-panfrost-v1-0-9ed8ca980e21@cherry.de/
diff mbox series

Patch

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index f49d7317ee9..67f392161c4 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -328,6 +328,7 @@  RECIPE_MAINTAINER:pn-libcap-ng = "Yi Zhao <yi.zhao@windriver.com>"
 RECIPE_MAINTAINER:pn-libcap-ng-python = "Yi Zhao <yi.zhao@windriver.com>"
 RECIPE_MAINTAINER:pn-libcgroup = "Unassigned <unassigned@yoctoproject.org>"
 RECIPE_MAINTAINER:pn-libcheck = "Yi Zhao <yi.zhao@windriver.com>"
+RECIPE_MAINTAINER:pn-libclc = "Khem Raj <raj.khem@gmail.com>"
 RECIPE_MAINTAINER:pn-libcomps = "Unassigned <unassigned@yoctoproject.org>"
 RECIPE_MAINTAINER:pn-libconvert-asn1-perl = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-libcxx = "Khem Raj <raj.khem@gmail.com>"
diff --git a/meta/recipes-devtools/clang/clang_git.bb b/meta/recipes-devtools/clang/clang_git.bb
index 1daac48edaf..4cbe22f4546 100644
--- a/meta/recipes-devtools/clang/clang_git.bb
+++ b/meta/recipes-devtools/clang/clang_git.bb
@@ -130,7 +130,7 @@  HF[vardepvalue] = "${HF}"
 
 # Ensure that LLVM_PROJECTS does not contain compiler runtime components e.g. libcxx etc
 # they are enabled via LLVM_ENABLE_RUNTIMES
-LLVM_PROJECTS ?= "clang;clang-tools-extra;libclc;lld"
+LLVM_PROJECTS ?= "clang;clang-tools-extra;lld"
 
 # linux hosts (.so) on Windows .pyd
 SOLIBSDEV:mingw32 = ".pyd"
@@ -191,7 +191,7 @@  EXTRA_OECMAKE:append:class-target = "\
 
 DEPENDS = "binutils zlib zstd libffi libxml2 libxml2-native ninja-native swig-native spirv-tools-native llvm-tblgen-native"
 DEPENDS:append:class-nativesdk = " clang-crosssdk-${SDK_SYS} virtual/nativesdk-cross-binutils nativesdk-python3"
-DEPENDS:append:class-target = " clang-cross-${TARGET_ARCH} python3 ${@bb.utils.contains('TC_CXX_RUNTIME', 'llvm', 'compiler-rt libcxx', '', d)} spirv-llvm-translator-native"
+DEPENDS:append:class-target = " clang-cross-${TARGET_ARCH} python3 ${@bb.utils.contains('TC_CXX_RUNTIME', 'llvm', 'compiler-rt libcxx', '', d)}"
 
 RRECOMMENDS:${PN} = "binutils"
 RRECOMMENDS:${PN}:append:class-target = "${@bb.utils.contains('TC_CXX_RUNTIME', 'llvm', ' libcxx-dev', '', d)}"
@@ -258,7 +258,6 @@  do_install:append:class-native () {
         install -Dm 0755 ${B}${BINPATHPREFIX}/bin/clangd-indexer ${D}${bindir}/clangd-indexer
     fi
     install -Dm 0755 ${B}${BINPATHPREFIX}/bin/clang-tidy-confusable-chars-gen ${D}${bindir}/clang-tidy-confusable-chars-gen
-    install -Dm 0755 ${B}${BINPATHPREFIX}/bin/prepare_builtins ${D}${bindir}/prepare_builtins
 
     for f in `find ${D}${bindir} -executable -type f -not -type l`; do
         test -n "`file -b $f|grep -i ELF`" && ${STRIP} $f
@@ -294,11 +293,11 @@  do_install:append:class-nativesdk () {
     fi
 }
 
-PROVIDES:append:class-native = " llvm-native libclc-native"
-PROVIDES:append:class-target = " llvm libclc"
-PROVIDES:append:class-nativesdk = " nativesdk-llvm nativesdk-libclc"
+PROVIDES:append:class-native = " llvm-native"
+PROVIDES:append:class-target = " llvm"
+PROVIDES:append:class-nativesdk = " nativesdk-llvm"
 
-PACKAGES =+ "${PN}-libllvm ${PN}-libclang-python ${PN}-libclang-cpp ${PN}-tidy ${PN}-format ${PN}-tools ${PN}-clc \
+PACKAGES =+ "${PN}-libllvm ${PN}-libclang-python ${PN}-libclang-cpp ${PN}-tidy ${PN}-format ${PN}-tools \
              libclang llvm-linker-tools"
 
 BBCLASSEXTEND = "native nativesdk"
@@ -315,8 +314,6 @@  RDEPENDS:${PN}-tools += "\
   perl-module-term-ansicolor \
 "
 
-RPROVIDES:${PN}-clc = "${MLPREFIX}libclc"
-
 RRECOMMENDS:${PN}-tidy += "${PN}-tools"
 
 FILES:llvm-linker-tools = "${libdir}/LLVMgold* ${libdir}/libLTO.so.* ${libdir}/LLVMPolly*"
@@ -383,8 +380,6 @@  FILES:${PN} += "\
   ${nonarch_libdir}/${BPN}/*/include/ \
 "
 
-FILES:${PN}-clc += "${datadir}/clc"
-
 FILES:${PN}-libllvm =+ "\
   ${libdir}/libLLVM.so.${MAJOR_VER}.${MINOR_VER} \
   ${libdir}/libLLVM-${MAJOR_VER}.so \
diff --git a/meta/recipes-devtools/clang/libclc_git.bb b/meta/recipes-devtools/clang/libclc_git.bb
new file mode 100644
index 00000000000..da849ba6ce8
--- /dev/null
+++ b/meta/recipes-devtools/clang/libclc_git.bb
@@ -0,0 +1,54 @@ 
+SUMMARY = "Implementation of the library requirements of the OpenCL C programming language."
+HOMEPAGE = "https://libclc.llvm.org"
+SECTION = "devel"
+
+require common-clang.inc
+require common-source.inc
+
+LIC_FILES_CHKSUM = "file://libclc/LICENSE.TXT;md5=7cc795f6cbb2d801d84336b83c8017db"
+
+inherit cmake pkgconfig
+
+TOOLCHAIN = "clang"
+
+# The clang dependency is actually just target LLVM
+DEPENDS += "clang spirv-llvm-translator-native"
+
+OECMAKE_SOURCEPATH = "${S}/libclc"
+
+# Semicolon-separated list of targets to build
+LIBCLC_TARGETS ?= "all"
+
+EXTRA_OECMAKE = "\
+    -DLLVM_DIR=${STAGING_LIBDIR}/cmake/llvm/ \
+    -DLIBCLC_TARGETS_TO_BUILD=${LIBCLC_TARGETS} \
+"
+
+# Until the target LLVMConfig.cmake points to the correct native directory for
+# the binaries, we need to specify their location manually.
+EXTRA_OECMAKE:append:class-target = "\
+    -DLLVM_TOOL_clang=${STAGING_BINDIR_TOOLCHAIN}/${HOST_PREFIX}clang \
+    -DLLVM_TOOL_llvm-as=${STAGING_BINDIR_TOOLCHAIN}/${HOST_PREFIX}llvm-as \
+    -DLLVM_TOOL_llvm-link=${STAGING_BINDIR_NATIVE}/llvm-link \
+    -DLLVM_TOOL_opt=${STAGING_BINDIR_NATIVE}/opt \
+"
+
+# Need to build a native prepare_builtins binary in target builds. The easiest
+# way to do this is with a second native cmake build tree.
+PATH .= ":${B}/native"
+
+do_build_prepare_builtins() {
+    cmake -G Ninja \
+        -S ${OECMAKE_SOURCEPATH} -B ${B}/native/ \
+        -DCMAKE_TOOLCHAIN_FILE:FILEPATH=${WORKDIR}/toolchain-native.cmake \
+        -DLIBCLC_TARGETS_TO_BUILD=
+    cmake --build ${B}/native --target prepare_builtins
+}
+do_build_prepare_builtins:class-native() {
+    :
+}
+addtask build_prepare_builtins after do_configure before do_compile
+
+FILES:${PN} += "${datadir}/clc"
+
+BBCLASSEXTEND = "native nativesdk"