diff mbox series

[2/5] toolchain: Provide abstraction for recipe specific toolchain selection

Message ID 20250612135955.1413300-2-richard.purdie@linuxfoundation.org
State Accepted, archived
Commit 45bdedd213aff8df3214b95ef2a8551c0abd93a0
Headers show
Series [1/5] base: Add deferred class event handler | expand

Commit Message

Richard Purdie June 12, 2025, 1:59 p.m. UTC
This implements a toolchain selection mechanism, defaulting to gcc
as per the existing defaults.

Introduce a variable called TOOLCHAIN, which denotes the familiar
name for toolchain e.g. "gcc" which selects GNU compiler + binutils
as default C/C++ toolchain or "clang" which will use LLVM/Clang Compiler

TOOLCHAIN variable has a global fallback to "gcc" in configuration
metadata. A distro can switch to using say "clang" as default system
compiler by defining

TOOLCHAIN ?= "clang"

In local.conf or other distro specific global configuration metadata

It is also selectable at recipe scope, since not all packages are
buildable with either clang or gcc, a recipe can explicitly demand
a given toolchain e.g. glibc can not be built with clang therefore
glibc recipe sets.

TOOLCHAIN = "gcc"

Based on ideas/work by Khem Raj <raj.khem@gmail.com>

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes-global/base.bbclass                         | 9 +++++++++
 .../clang.inc => classes/toolchain/clang.bbclass}        | 0
 .../toolchain/gcc-native.bbclass}                        | 0
 .../toolchain/gcc.inc => classes/toolchain/gcc.bbclass}  | 0
 meta/conf/bitbake.conf                                   | 3 ---
 meta/recipes-devtools/binutils/binutils-cross.inc        | 2 ++
 meta/recipes-devtools/clang/clang-cross_git.bb           | 1 +
 meta/recipes-devtools/clang/clang-crosssdk_git.bb        | 1 +
 8 files changed, 13 insertions(+), 3 deletions(-)
 rename meta/{conf/toolchain/clang.inc => classes/toolchain/clang.bbclass} (100%)
 rename meta/{conf/toolchain/build-gcc.inc => classes/toolchain/gcc-native.bbclass} (100%)
 rename meta/{conf/toolchain/gcc.inc => classes/toolchain/gcc.bbclass} (100%)

Comments

Peter Kjellerstedt June 17, 2025, 9:12 a.m. UTC | #1
> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Richard Purdie via lists.openembedded.org
> Sent: den 12 juni 2025 16:00
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH 2/5] toolchain: Provide abstraction for recipe
> specific toolchain selection
> 
> This implements a toolchain selection mechanism, defaulting to gcc
> as per the existing defaults.
> 
> Introduce a variable called TOOLCHAIN, which denotes the familiar
> name for toolchain e.g. "gcc" which selects GNU compiler + binutils
> as default C/C++ toolchain or "clang" which will use LLVM/Clang Compiler
> 
> TOOLCHAIN variable has a global fallback to "gcc" in configuration
> metadata. A distro can switch to using say "clang" as default system
> compiler by defining
> 
> TOOLCHAIN ?= "clang"
> 
> In local.conf or other distro specific global configuration metadata
> 
> It is also selectable at recipe scope, since not all packages are
> buildable with either clang or gcc, a recipe can explicitly demand
> a given toolchain e.g. glibc can not be built with clang therefore
> glibc recipe sets.
> 
> TOOLCHAIN = "gcc"
> 
> Based on ideas/work by Khem Raj <raj.khem@gmail.com>
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes-global/base.bbclass                         | 9 +++++++++
>  .../clang.inc => classes/toolchain/clang.bbclass}        | 0
>  .../toolchain/gcc-native.bbclass}                        | 0
>  .../toolchain/gcc.inc => classes/toolchain/gcc.bbclass}  | 0
>  meta/conf/bitbake.conf                                   | 3 ---
>  meta/recipes-devtools/binutils/binutils-cross.inc        | 2 ++
>  meta/recipes-devtools/clang/clang-cross_git.bb           | 1 +
>  meta/recipes-devtools/clang/clang-crosssdk_git.bb        | 1 +
>  8 files changed, 13 insertions(+), 3 deletions(-)
>  rename meta/{conf/toolchain/clang.inc => classes/toolchain/clang.bbclass} (100%)
>  rename meta/{conf/toolchain/build-gcc.inc => classes/toolchain/gcc-native.bbclass} (100%)
>  rename meta/{conf/toolchain/gcc.inc => classes/toolchain/gcc.bbclass} (100%)
> 
> diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
> index 4ac3b83eb5c..4809cca27de 100644
> --- a/meta/classes-global/base.bbclass
> +++ b/meta/classes-global/base.bbclass
> @@ -19,6 +19,15 @@ PACKAGECONFIG_CONFARGS ??= ""
> 
>  inherit metadata_scm
> 
> +TOOLCHAIN ??= "gcc"
> +TOOLCHAIN:class-native ??= "gcc"
> +TOOLCHAIN:class-cross ??= "gcc"
> +TOOLCHAIN:class-crosssdk ??= "gcc"
> +TOOLCHAIN:class-nativesdk ??= "gcc"
> +
> +inherit toolchain/gcc-native
> +inherit_defer toolchain/${TOOLCHAIN}

What if you want to use clang as your native compiler for everything, 
i.e., even when building target recipes? Would something like this 
make sense?

TOOLCHAIN_NATIVE ??= "gcc"

TOOLCHAIN ??= "gcc"
TOOLCHAIN:class-native ??= "${TOOLCHAIN_NATIVE}"
TOOLCHAIN:class-cross ??= "gcc"
TOOLCHAIN:class-crosssdk ??= "gcc"
TOOLCHAIN:class-nativesdk ??= "gcc"

inherit toolchain/${TOOLCHAIN_NATIVE}-native
inherit_defer toolchain/${TOOLCHAIN}

//Peter
Richard Purdie June 17, 2025, 9:21 a.m. UTC | #2
On Tue, 2025-06-17 at 09:12 +0000, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Richard Purdie via lists.openembedded.org
> > Sent: den 12 juni 2025 16:00
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH 2/5] toolchain: Provide abstraction for recipe
> > specific toolchain selection
> > 
> > This implements a toolchain selection mechanism, defaulting to gcc
> > as per the existing defaults.
> > 
> > Introduce a variable called TOOLCHAIN, which denotes the familiar
> > name for toolchain e.g. "gcc" which selects GNU compiler + binutils
> > as default C/C++ toolchain or "clang" which will use LLVM/Clang Compiler
> > 
> > TOOLCHAIN variable has a global fallback to "gcc" in configuration
> > metadata. A distro can switch to using say "clang" as default system
> > compiler by defining
> > 
> > TOOLCHAIN ?= "clang"
> > 
> > In local.conf or other distro specific global configuration metadata
> > 
> > It is also selectable at recipe scope, since not all packages are
> > buildable with either clang or gcc, a recipe can explicitly demand
> > a given toolchain e.g. glibc can not be built with clang therefore
> > glibc recipe sets.
> > 
> > TOOLCHAIN = "gcc"
> > 
> > Based on ideas/work by Khem Raj <raj.khem@gmail.com>
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> >  meta/classes-global/base.bbclass                         | 9 +++++++++
> >  .../clang.inc => classes/toolchain/clang.bbclass}        | 0
> >  .../toolchain/gcc-native.bbclass}                        | 0
> >  .../toolchain/gcc.inc => classes/toolchain/gcc.bbclass}  | 0
> >  meta/conf/bitbake.conf                                   | 3 ---
> >  meta/recipes-devtools/binutils/binutils-cross.inc        | 2 ++
> >  meta/recipes-devtools/clang/clang-cross_git.bb           | 1 +
> >  meta/recipes-devtools/clang/clang-crosssdk_git.bb        | 1 +
> >  8 files changed, 13 insertions(+), 3 deletions(-)
> >  rename meta/{conf/toolchain/clang.inc => classes/toolchain/clang.bbclass} (100%)
> >  rename meta/{conf/toolchain/build-gcc.inc => classes/toolchain/gcc-native.bbclass} (100%)
> >  rename meta/{conf/toolchain/gcc.inc => classes/toolchain/gcc.bbclass} (100%)
> > 
> > diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
> > index 4ac3b83eb5c..4809cca27de 100644
> > --- a/meta/classes-global/base.bbclass
> > +++ b/meta/classes-global/base.bbclass
> > @@ -19,6 +19,15 @@ PACKAGECONFIG_CONFARGS ??= ""
> > 
> >  inherit metadata_scm
> > 
> > +TOOLCHAIN ??= "gcc"
> > +TOOLCHAIN:class-native ??= "gcc"
> > +TOOLCHAIN:class-cross ??= "gcc"
> > +TOOLCHAIN:class-crosssdk ??= "gcc"
> > +TOOLCHAIN:class-nativesdk ??= "gcc"
> > +
> > +inherit toolchain/gcc-native
> > +inherit_defer toolchain/${TOOLCHAIN}
> 
> What if you want to use clang as your native compiler for everything,
> i.e., even when building target recipes? Would something like this 
> make sense?
> 
> TOOLCHAIN_NATIVE ??= "gcc"
> 
> TOOLCHAIN ??= "gcc"
> TOOLCHAIN:class-native ??= "${TOOLCHAIN_NATIVE}"
> TOOLCHAIN:class-cross ??= "gcc"
> TOOLCHAIN:class-crosssdk ??= "gcc"
> TOOLCHAIN:class-nativesdk ??= "gcc"
> 
> inherit toolchain/${TOOLCHAIN_NATIVE}-native
> inherit_defer toolchain/${TOOLCHAIN}

You'd need to use inherit_defer there ;-)

But, I know what you mean and this doesn't work since a recipe like
clang-cross sets TOOLCHAIN = "clang" and you'd override it here to gcc.

Ultimately we do want to allow native and sdk toolchain selection but
I'd not got to enabling that yet.

Cheers,

Richard
Richard Purdie June 17, 2025, 9:32 a.m. UTC | #3
On Tue, 2025-06-17 at 10:21 +0100, Richard Purdie via
lists.openembedded.org wrote:
> On Tue, 2025-06-17 at 09:12 +0000, Peter Kjellerstedt wrote:
> > > -----Original Message-----
> > > From:
> > > openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org
> > > > On Behalf Of Richard Purdie via lists.openembedded.org
> > > Sent: den 12 juni 2025 16:00
> > > To: openembedded-core@lists.openembedded.org
> > > Subject: [OE-core] [PATCH 2/5] toolchain: Provide abstraction for
> > > recipe
> > > specific toolchain selection
> > > 
> > > This implements a toolchain selection mechanism, defaulting to
> > > gcc
> > > as per the existing defaults.
> > > 
> > > Introduce a variable called TOOLCHAIN, which denotes the familiar
> > > name for toolchain e.g. "gcc" which selects GNU compiler +
> > > binutils
> > > as default C/C++ toolchain or "clang" which will use LLVM/Clang
> > > Compiler
> > > 
> > > TOOLCHAIN variable has a global fallback to "gcc" in
> > > configuration
> > > metadata. A distro can switch to using say "clang" as default
> > > system
> > > compiler by defining
> > > 
> > > TOOLCHAIN ?= "clang"
> > > 
> > > In local.conf or other distro specific global configuration
> > > metadata
> > > 
> > > It is also selectable at recipe scope, since not all packages are
> > > buildable with either clang or gcc, a recipe can explicitly
> > > demand
> > > a given toolchain e.g. glibc can not be built with clang
> > > therefore
> > > glibc recipe sets.
> > > 
> > > TOOLCHAIN = "gcc"
> > > 
> > > Based on ideas/work by Khem Raj <raj.khem@gmail.com>
> > > 
> > > Signed-off-by: Richard Purdie
> > > <richard.purdie@linuxfoundation.org>
> > > ---
> > >  meta/classes-global/base.bbclass                         | 9
> > > +++++++++
> > >  .../clang.inc => classes/toolchain/clang.bbclass}        | 0
> > >  .../toolchain/gcc-native.bbclass}                        | 0
> > >  .../toolchain/gcc.inc => classes/toolchain/gcc.bbclass}  | 0
> > >  meta/conf/bitbake.conf                                   | 3 ---
> > >  meta/recipes-devtools/binutils/binutils-cross.inc        | 2 ++
> > >  meta/recipes-devtools/clang/clang-cross_git.bb           | 1 +
> > >  meta/recipes-devtools/clang/clang-crosssdk_git.bb        | 1 +
> > >  8 files changed, 13 insertions(+), 3 deletions(-)
> > >  rename meta/{conf/toolchain/clang.inc =>
> > > classes/toolchain/clang.bbclass} (100%)
> > >  rename meta/{conf/toolchain/build-gcc.inc =>
> > > classes/toolchain/gcc-native.bbclass} (100%)
> > >  rename meta/{conf/toolchain/gcc.inc =>
> > > classes/toolchain/gcc.bbclass} (100%)
> > > 
> > > diff --git a/meta/classes-global/base.bbclass b/meta/classes-
> > > global/base.bbclass
> > > index 4ac3b83eb5c..4809cca27de 100644
> > > --- a/meta/classes-global/base.bbclass
> > > +++ b/meta/classes-global/base.bbclass
> > > @@ -19,6 +19,15 @@ PACKAGECONFIG_CONFARGS ??= ""
> > > 
> > >  inherit metadata_scm
> > > 
> > > +TOOLCHAIN ??= "gcc"
> > > +TOOLCHAIN:class-native ??= "gcc"
> > > +TOOLCHAIN:class-cross ??= "gcc"
> > > +TOOLCHAIN:class-crosssdk ??= "gcc"
> > > +TOOLCHAIN:class-nativesdk ??= "gcc"
> > > +
> > > +inherit toolchain/gcc-native
> > > +inherit_defer toolchain/${TOOLCHAIN}
> > 
> > What if you want to use clang as your native compiler for
> > everything,
> > i.e., even when building target recipes? Would something like this 
> > make sense?
> > 
> > TOOLCHAIN_NATIVE ??= "gcc"
> > 
> > TOOLCHAIN ??= "gcc"
> > TOOLCHAIN:class-native ??= "${TOOLCHAIN_NATIVE}"
> > TOOLCHAIN:class-cross ??= "gcc"
> > TOOLCHAIN:class-crosssdk ??= "gcc"
> > TOOLCHAIN:class-nativesdk ??= "gcc"
> > 
> > inherit toolchain/${TOOLCHAIN_NATIVE}-native
> > inherit_defer toolchain/${TOOLCHAIN}
> 
> You'd need to use inherit_defer there ;-)
> 
> But, I know what you mean and this doesn't work since a recipe like
> clang-cross sets TOOLCHAIN = "clang" and you'd override it here to
> gcc.
> 
> Ultimately we do want to allow native and sdk toolchain selection but
> I'd not got to enabling that yet.

I also just realised this was against an older patch. I did end up
merging something different to this due to the reasons I mentioned. I
don't think it was possible to use just the TOOLCHAIN variable to cover
all options, much as I'd liked to have.

Cheers,

Richard
diff mbox series

Patch

diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 4ac3b83eb5c..4809cca27de 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -19,6 +19,15 @@  PACKAGECONFIG_CONFARGS ??= ""
 
 inherit metadata_scm
 
+TOOLCHAIN ??= "gcc"
+TOOLCHAIN:class-native ??= "gcc"
+TOOLCHAIN:class-cross ??= "gcc"
+TOOLCHAIN:class-crosssdk ??= "gcc"
+TOOLCHAIN:class-nativesdk ??= "gcc"
+
+inherit toolchain/gcc-native
+inherit_defer toolchain/${TOOLCHAIN}
+
 def lsb_distro_identifier(d):
     adjust = d.getVar('LSB_DISTRO_ADJUST')
     adjust_func = None
diff --git a/meta/conf/toolchain/clang.inc b/meta/classes/toolchain/clang.bbclass
similarity index 100%
rename from meta/conf/toolchain/clang.inc
rename to meta/classes/toolchain/clang.bbclass
diff --git a/meta/conf/toolchain/build-gcc.inc b/meta/classes/toolchain/gcc-native.bbclass
similarity index 100%
rename from meta/conf/toolchain/build-gcc.inc
rename to meta/classes/toolchain/gcc-native.bbclass
diff --git a/meta/conf/toolchain/gcc.inc b/meta/classes/toolchain/gcc.bbclass
similarity index 100%
rename from meta/conf/toolchain/gcc.inc
rename to meta/classes/toolchain/gcc.bbclass
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e600d9d774a..9a91d6aa6a4 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -834,9 +834,6 @@  include conf/licenses.conf
 require conf/sanity.conf
 include conf/bblock.conf
 
-require toolchain/gcc.inc
-require toolchain/build-gcc.inc
-
 ##################################################################
 # Weak variables (usually to retain backwards compatibility)
 ##################################################################
diff --git a/meta/recipes-devtools/binutils/binutils-cross.inc b/meta/recipes-devtools/binutils/binutils-cross.inc
index 9c371e7e137..c545ea2ad97 100644
--- a/meta/recipes-devtools/binutils/binutils-cross.inc
+++ b/meta/recipes-devtools/binutils/binutils-cross.inc
@@ -9,6 +9,8 @@  TARGET_ARCH[vardepvalue] = "${TARGET_ARCH}"
 INHIBIT_DEFAULT_DEPS = "1"
 INHIBIT_AUTOTOOLS_DEPS = "1"
 
+TOOLCHAIN = "gcc"
+
 SRC_URI += "file://0002-binutils-cross-Do-not-generate-linker-script-directo.patch"
 
 # Specify lib-path else we use a load of search dirs which we don't use
diff --git a/meta/recipes-devtools/clang/clang-cross_git.bb b/meta/recipes-devtools/clang/clang-cross_git.bb
index 9b9b120a3d7..323cc0d8809 100644
--- a/meta/recipes-devtools/clang/clang-cross_git.bb
+++ b/meta/recipes-devtools/clang/clang-cross_git.bb
@@ -11,6 +11,7 @@  PN = "clang-cross-${TARGET_ARCH}"
 require common-clang.inc
 require common-source.inc
 inherit cross
+TOOLCHAIN = "clang"
 DEPENDS += "clang-native virtual/cross-binutils"
 
 #INHIBIT_PACKAGE_STRIP = "1"
diff --git a/meta/recipes-devtools/clang/clang-crosssdk_git.bb b/meta/recipes-devtools/clang/clang-crosssdk_git.bb
index 47ac96f4f93..ef162ef153f 100644
--- a/meta/recipes-devtools/clang/clang-crosssdk_git.bb
+++ b/meta/recipes-devtools/clang/clang-crosssdk_git.bb
@@ -11,6 +11,7 @@  PN = "clang-crosssdk-${SDK_SYS}"
 require common-clang.inc
 require common-source.inc
 inherit crosssdk
+TOOLCHAIN = "clang"
 DEPENDS += "clang-native nativesdk-clang-glue virtual/nativesdk-cross-binutils virtual/nativesdk-libc"
 
 do_install() {