[v4] feature-arm-crypto: Add +nocrypto to -mcpu when crypto not in features

When crypto is not in tune features then add +nocryto to
-mcpu explicitly. This makes the behavior between clang
and gcc match. Currently -mcpu=cortex-a72 has different
behavior in clang and gcc in terms of what features are
considered default. Clang enables different set of common
features than gcc on other hand. For example clang
enables crypto with default set but gcc
does not, gcc recommends to disable unavailable extensions
in -mcpu [1] explicitly. crypto is optional on cortex-a53
and cortex-a72. This is not as common but Broadcom
SOCs in raspberrypi3/4 have dropped crypto for cost
reasons [2]. This results in illegal instruction
traps [3] [4] when building components e.g. chromium,
qtwebengine, weston etc. with clang using -mcpu=cortex-a72
for rpi4 target.

Adding +nocrypto makes clang behave like gcc does today. We
do have separate tune if crypto enabled cortex-a72 cores
are to be targeted (cortexa72-cryto) as DEFAULTTUNE

They are added to default feature file since crypto extension
is available in multiple arm architecture versions e.g. armv8,
armv9. It is optional extension as per spec [5]

Extensions can be enabled and disabled with -mcpu using the same
syntax as with -march, and have same effect thats why it is
intrumented via TUNE_CCARGS_MARCH_OPTS

[1] https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html#index-mcpu-2
[2] https://forums.raspberrypi.com/viewtopic.php?f=63&t=207888#p1332960
[3] https://github.com/llvm/llvm-project/issues/85699
[4] https://github.com/llvm/llvm-project/issues/90365
[5] https://developer.arm.com/documentation/109697/2025_06/Feature-descriptions/The-Armv9-0-architecture-extension

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Quentin Schulz <quentin.schulz@cherry.de>
---
v2: Move the change to feature-arm-crypto.inc.
v3: Adjust commit message for clarity on reason to be in common feature file.
v4: Address commit msg comments, add section on why use TUNE_CCARGS_MARCH_OPTS

 meta/conf/machine/include/arm/feature-arm-crypto.inc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Hi Khem

On 7/24/25 7:06 PM, Khem Raj wrote:
> When crypto is not in tune features then add +nocryto to
> -mcpu explicitly. This makes the behavior between clang
> and gcc match. Currently -mcpu=cortex-a72 has different
> behavior in clang and gcc in terms of what features are
> considered default. Clang enables different set of common
> features than gcc on other hand. For example clang
> enables crypto with default set but gcc
> does not, gcc recommends to disable unavailable extensions
> in -mcpu [1] explicitly. crypto is optional on cortex-a53
> and cortex-a72. This is not as common but Broadcom
> SOCs in raspberrypi3/4 have dropped crypto for cost
> reasons [2]. This results in illegal instruction
> traps [3] [4] when building components e.g. chromium,
> qtwebengine, weston etc. with clang using -mcpu=cortex-a72
> for rpi4 target.
> 
> Adding +nocrypto makes clang behave like gcc does today. We
> do have separate tune if crypto enabled cortex-a72 cores
> are to be targeted (cortexa72-cryto) as DEFAULTTUNE
> 
> They are added to default feature file since crypto extension
> is available in multiple arm architecture versions e.g. armv8,
> armv9. It is optional extension as per spec [5]
> 
> Extensions can be enabled and disabled with -mcpu using the same
> syntax as with -march, and have same effect thats why it is
> intrumented via TUNE_CCARGS_MARCH_OPTS
> 

I believe this is incorrect. TUNE_CCARGS_MARCH_OPTS applies to -march, 
not -mcpu.

If that is correct, then The whole explanation should be targeting march 
and not mcpu as we're changing march and not mcpu (though I assume they 
trickle from march to mcpu inside the toolchain).

Cheers,
Quentin

On Thu, Jul 24, 2025 at 10:10 AM Quentin Schulz
<quentin.schulz@cherry.de> wrote:
>
> Hi Khem
>
> On 7/24/25 7:06 PM, Khem Raj wrote:
> > When crypto is not in tune features then add +nocryto to
> > -mcpu explicitly. This makes the behavior between clang
> > and gcc match. Currently -mcpu=cortex-a72 has different
> > behavior in clang and gcc in terms of what features are
> > considered default. Clang enables different set of common
> > features than gcc on other hand. For example clang
> > enables crypto with default set but gcc
> > does not, gcc recommends to disable unavailable extensions
> > in -mcpu [1] explicitly. crypto is optional on cortex-a53
> > and cortex-a72. This is not as common but Broadcom
> > SOCs in raspberrypi3/4 have dropped crypto for cost
> > reasons [2]. This results in illegal instruction
> > traps [3] [4] when building components e.g. chromium,
> > qtwebengine, weston etc. with clang using -mcpu=cortex-a72
> > for rpi4 target.
> >
> > Adding +nocrypto makes clang behave like gcc does today. We
> > do have separate tune if crypto enabled cortex-a72 cores
> > are to be targeted (cortexa72-cryto) as DEFAULTTUNE
> >
> > They are added to default feature file since crypto extension
> > is available in multiple arm architecture versions e.g. armv8,
> > armv9. It is optional extension as per spec [5]
> >
> > Extensions can be enabled and disabled with -mcpu using the same
> > syntax as with -march, and have same effect thats why it is
> > intrumented via TUNE_CCARGS_MARCH_OPTS
> >
>
> I believe this is incorrect. TUNE_CCARGS_MARCH_OPTS applies to -march,
> not -mcpu.

Well, it applied to both in cpu specific tune files ( e.g.
tune-cortexa55.inc ) we override
arch specific TUNE_FEATURES and replace with then CPU specific TUNE_FEATURES
 This results in  TUNE_CCARGS_MARCH being empty and TUNE_CCARGS_MARCH_OPTS
are still valid and they can appended to TUNE_CCARGS

in conf/machine/include/arm/arch-arm.inc we have
TUNE_CCARGS .= "${TUNE_CCARGS_MARCH}${TUNE_CCARGS_MARCH_OPTS
}"

Due to include order this append appears after the TUNE_CCARGS is
appended with -mcpu option in a mcpu tune file e.g. tune-cortexa55.inc

TUNE_CCARGS .= "${@bb.utils.contains('TUNE_FEATURES', 'cortexa55', '
-mcpu=cortex-a55', '', d)}"

so in the end we get -mcpu=cortex-a55+nocrypto or
-mcpu=cortex-a55+crypto in this case
if you were optimizing widely for an arch then the effect will be via
TUNE_CCARGS_MARCH and
TUNE_CCARGS_MARCH_OPTS

I do wonder whether renaming both TUNE_CCARGS_MARCH and
TUNE_CCARGS_MARCH_OPTS to something more generic could help lessen the
confusion. It's there because we have -march and
introduced -mcpu options later on.

>
> If that is correct, then The whole explanation should be targeting march
> and not mcpu as we're changing march and not mcpu (though I assume they
> trickle from march to mcpu inside the toolchain).
>

your assumption is right. Although, now thinking of this a bit more. I
think we can rename the
TUNE_CCARGS_MARCH_OPTS as I suggested in prior email to something like
TUNE_CCARGS_CPU_EXTENSIONS but in the end they are march options from
gcc/clang's
perspective, its just they are instrumented via either march or mcpu

hope that helps.
> Cheers,
> Quentin

Hi Khem,

On 7/24/25 8:32 PM, Khem Raj via lists.openembedded.org wrote:
> On Thu, Jul 24, 2025 at 10:10 AM Quentin Schulz
> <quentin.schulz@cherry.de> wrote:
>>
>> Hi Khem
>>
>> On 7/24/25 7:06 PM, Khem Raj wrote:
>>> When crypto is not in tune features then add +nocryto to
>>> -mcpu explicitly. This makes the behavior between clang
>>> and gcc match. Currently -mcpu=cortex-a72 has different
>>> behavior in clang and gcc in terms of what features are
>>> considered default. Clang enables different set of common
>>> features than gcc on other hand. For example clang
>>> enables crypto with default set but gcc
>>> does not, gcc recommends to disable unavailable extensions
>>> in -mcpu [1] explicitly. crypto is optional on cortex-a53
>>> and cortex-a72. This is not as common but Broadcom
>>> SOCs in raspberrypi3/4 have dropped crypto for cost
>>> reasons [2]. This results in illegal instruction
>>> traps [3] [4] when building components e.g. chromium,
>>> qtwebengine, weston etc. with clang using -mcpu=cortex-a72
>>> for rpi4 target.
>>>
>>> Adding +nocrypto makes clang behave like gcc does today. We
>>> do have separate tune if crypto enabled cortex-a72 cores
>>> are to be targeted (cortexa72-cryto) as DEFAULTTUNE
>>>
>>> They are added to default feature file since crypto extension
>>> is available in multiple arm architecture versions e.g. armv8,
>>> armv9. It is optional extension as per spec [5]
>>>
>>> Extensions can be enabled and disabled with -mcpu using the same
>>> syntax as with -march, and have same effect thats why it is
>>> intrumented via TUNE_CCARGS_MARCH_OPTS
>>>
>>
>> I believe this is incorrect. TUNE_CCARGS_MARCH_OPTS applies to -march,
>> not -mcpu.
> 
> Well, it applied to both in cpu specific tune files ( e.g.
> tune-cortexa55.inc ) we override
> arch specific TUNE_FEATURES and replace with then CPU specific TUNE_FEATURES
>   This results in  TUNE_CCARGS_MARCH being empty and TUNE_CCARGS_MARCH_OPTS
> are still valid and they can appended to TUNE_CCARGS
> 
> in conf/machine/include/arm/arch-arm.inc we have
> TUNE_CCARGS .= "${TUNE_CCARGS_MARCH}${TUNE_CCARGS_MARCH_OPTS
> }"
> 
> Due to include order this append appears after the TUNE_CCARGS is
> appended with -mcpu option in a mcpu tune file e.g. tune-cortexa55.inc
> 
> TUNE_CCARGS .= "${@bb.utils.contains('TUNE_FEATURES', 'cortexa55', '
> -mcpu=cortex-a55', '', d)}"
> 
> so in the end we get -mcpu=cortex-a55+nocrypto or
> -mcpu=cortex-a55+crypto in this case
> if you were optimizing widely for an arch then the effect will be via
> TUNE_CCARGS_MARCH and
> TUNE_CCARGS_MARCH_OPTS
> 

And there's no world where we could have both cortexa55 AND armv8-a in 
TUNE_FEATURES? I assume this doesn't make sense? If it does, then we 
could have -mcpu without the +crypto and -march with the +crypto, I 
assume this could be an issue? I'm probably overthinking this though, 
aren't I?

> I do wonder whether renaming both TUNE_CCARGS_MARCH and
> TUNE_CCARGS_MARCH_OPTS to something more generic could help lessen the
> confusion. It's there because we have -march and
> introduced -mcpu options later on.
> 

It depends. If there are some scenario where we can have both -mcpu and 
-march specified, then:
- if they can have different options (I doubt this is legal), then each 
have their own _OPTS is desirable,
- if they cannot, then the options likely need to be repeated for both 
mcpu and march

If we can have one or the other (mcpu/march), then renaming the variable 
so its name is more generic would be nice indeed :)

Cheers,
Quentin

On Fri, Aug 1, 2025 at 3:49 AM Quentin Schulz <quentin.schulz@cherry.de> wrote:
>
> Hi Khem,
>
> On 7/24/25 8:32 PM, Khem Raj via lists.openembedded.org wrote:
> > On Thu, Jul 24, 2025 at 10:10 AM Quentin Schulz
> > <quentin.schulz@cherry.de> wrote:
> >>
> >> Hi Khem
> >>
> >> On 7/24/25 7:06 PM, Khem Raj wrote:
> >>> When crypto is not in tune features then add +nocryto to
> >>> -mcpu explicitly. This makes the behavior between clang
> >>> and gcc match. Currently -mcpu=cortex-a72 has different
> >>> behavior in clang and gcc in terms of what features are
> >>> considered default. Clang enables different set of common
> >>> features than gcc on other hand. For example clang
> >>> enables crypto with default set but gcc
> >>> does not, gcc recommends to disable unavailable extensions
> >>> in -mcpu [1] explicitly. crypto is optional on cortex-a53
> >>> and cortex-a72. This is not as common but Broadcom
> >>> SOCs in raspberrypi3/4 have dropped crypto for cost
> >>> reasons [2]. This results in illegal instruction
> >>> traps [3] [4] when building components e.g. chromium,
> >>> qtwebengine, weston etc. with clang using -mcpu=cortex-a72
> >>> for rpi4 target.
> >>>
> >>> Adding +nocrypto makes clang behave like gcc does today. We
> >>> do have separate tune if crypto enabled cortex-a72 cores
> >>> are to be targeted (cortexa72-cryto) as DEFAULTTUNE
> >>>
> >>> They are added to default feature file since crypto extension
> >>> is available in multiple arm architecture versions e.g. armv8,
> >>> armv9. It is optional extension as per spec [5]
> >>>
> >>> Extensions can be enabled and disabled with -mcpu using the same
> >>> syntax as with -march, and have same effect thats why it is
> >>> intrumented via TUNE_CCARGS_MARCH_OPTS
> >>>
> >>
> >> I believe this is incorrect. TUNE_CCARGS_MARCH_OPTS applies to -march,
> >> not -mcpu.
> >
> > Well, it applied to both in cpu specific tune files ( e.g.
> > tune-cortexa55.inc ) we override
> > arch specific TUNE_FEATURES and replace with then CPU specific TUNE_FEATURES
> >   This results in  TUNE_CCARGS_MARCH being empty and TUNE_CCARGS_MARCH_OPTS
> > are still valid and they can appended to TUNE_CCARGS
> >
> > in conf/machine/include/arm/arch-arm.inc we have
> > TUNE_CCARGS .= "${TUNE_CCARGS_MARCH}${TUNE_CCARGS_MARCH_OPTS
> > }"
> >
> > Due to include order this append appears after the TUNE_CCARGS is
> > appended with -mcpu option in a mcpu tune file e.g. tune-cortexa55.inc
> >
> > TUNE_CCARGS .= "${@bb.utils.contains('TUNE_FEATURES', 'cortexa55', '
> > -mcpu=cortex-a55', '', d)}"
> >
> > so in the end we get -mcpu=cortex-a55+nocrypto or
> > -mcpu=cortex-a55+crypto in this case
> > if you were optimizing widely for an arch then the effect will be via
> > TUNE_CCARGS_MARCH and
> > TUNE_CCARGS_MARCH_OPTS
> >
>
> And there's no world where we could have both cortexa55 AND armv8-a in
> TUNE_FEATURES?

cortexa55 tune files overwrite TUNE_FEATURES and armv8a will be gone
from it. In the unlikely case of such inclusion
-march then becomes ineffective.

I assume this doesn't make sense? If it does, then we
> could have -mcpu without the +crypto and -march with the +crypto, I
> assume this could be an issue? I'm probably overthinking this though,
> aren't I?
>
> > I do wonder whether renaming both TUNE_CCARGS_MARCH and
> > TUNE_CCARGS_MARCH_OPTS to something more generic could help lessen the
> > confusion. It's there because we have -march and
> > introduced -mcpu options later on.
> >
>
> It depends. If there are some scenario where we can have both -mcpu and
> -march specified, then:
> - if they can have different options (I doubt this is legal), then each
> have their own _OPTS is desirable,
> - if they cannot, then the options likely need to be repeated for both
> mcpu and march
>

internally all these extensions translate on top of some -march, it
could come via
-march or -mcpu.


> If we can have one or the other (mcpu/march), then renaming the variable
> so its name is more generic would be nice indeed :)

Yeah, although a mechanical change, it must be done carefully.

>
> Cheers,
> Quentin

Hi

On Thu, Jul 24, 2025 at 7:06 PM Khem Raj via lists.openembedded.org
<raj.khem=gmail.com@lists.openembedded.org> wrote:
>
> When crypto is not in tune features then add +nocryto to
> -mcpu explicitly. This makes the behavior between clang
> and gcc match. Currently -mcpu=cortex-a72 has different
> behavior in clang and gcc in terms of what features are
> considered default. Clang enables different set of common
> features than gcc on other hand. For example clang
> enables crypto with default set but gcc
> does not, gcc recommends to disable unavailable extensions
> in -mcpu [1] explicitly. crypto is optional on cortex-a53
> and cortex-a72. This is not as common but Broadcom
> SOCs in raspberrypi3/4 have dropped crypto for cost
> reasons [2]. This results in illegal instruction
> traps [3] [4] when building components e.g. chromium,
> qtwebengine, weston etc. with clang using -mcpu=cortex-a72
> for rpi4 target.
>
> Adding +nocrypto makes clang behave like gcc does today. We
> do have separate tune if crypto enabled cortex-a72 cores
> are to be targeted (cortexa72-cryto) as DEFAULTTUNE
>
> They are added to default feature file since crypto extension
> is available in multiple arm architecture versions e.g. armv8,
> armv9. It is optional extension as per spec [5]
>
> Extensions can be enabled and disabled with -mcpu using the same
> syntax as with -march, and have same effect thats why it is
> intrumented via TUNE_CCARGS_MARCH_OPTS
>
> [1] https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html#index-mcpu-2
> [2] https://forums.raspberrypi.com/viewtopic.php?f=63&t=207888#p1332960
> [3] https://github.com/llvm/llvm-project/issues/85699
> [4] https://github.com/llvm/llvm-project/issues/90365
> [5] https://developer.arm.com/documentation/109697/2025_06/Feature-descriptions/The-Armv9-0-architecture-extension
>
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> Cc: Quentin Schulz <quentin.schulz@cherry.de>
> ---
> v2: Move the change to feature-arm-crypto.inc.
> v3: Adjust commit message for clarity on reason to be in common feature file.
> v4: Address commit msg comments, add section on why use TUNE_CCARGS_MARCH_OPTS
>
>  meta/conf/machine/include/arm/feature-arm-crypto.inc | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/meta/conf/machine/include/arm/feature-arm-crypto.inc b/meta/conf/machine/include/arm/feature-arm-crypto.inc
> index aade6ce08d5..8202de78e46 100644
> --- a/meta/conf/machine/include/arm/feature-arm-crypto.inc
> +++ b/meta/conf/machine/include/arm/feature-arm-crypto.inc
> @@ -2,4 +2,5 @@
>  # armv8-a, armv8.1-a, armv8.3-a, armv8.4-a, armv8.5-a, armv8.6-a, and armv8-r
>
>  TUNEVALID[crypto] = "Enable cryptographic instructions for ARMv8"
> -TUNE_CCARGS_MARCH_OPTS .= "${@bb.utils.contains('TUNE_FEATURES', 'crypto', '+crypto', '', d)}"
> +# See  https://github.com/llvm/llvm-project/issues/85699
> +TUNE_CCARGS_MARCH_OPTS .= "${@bb.utils.contains('TUNE_FEATURES', 'crypto', '+crypto', '+nocrypto', d)}"
>
> -=-=-=-=-=-=-=-=-=-=-=-

This breaks machines which set `DEFAULTTUNE ?= "aarch64"` and use gcc
as e.g. done for TI SoC [1].
For that tune the `TUNE_CCARGS_MARCH` is not set at all resulting in a
lone `+nocrypto`
in the compiler arguments. Note that for gcc additionally
`-march=aarch64` is not an allowed argument.

I don't really know what is a good way forward. Of course one could
not use aarch64 as a default tune
to work around the issue.

Any ideas?

Regards
Max

| checking for aarch64-tdx-linux-gcc... aarch64-tdx-linux-gcc
+nocrypto -mbranch-protection=standard -fstack-protector-strong  -O2
-D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security
--sysroot=...aarch64-tdx-linux/libgcc-initial/15.1.0/recipe-sysroot
| checking for suffix of object files... configure: error: in
`...aarch64-tdx-linux/libgcc-initial/15.1.0/gcc-15.1.0/build.aarch64-tdx-linux.aarch64-tdx-linux/libgcc':
| configure: error: cannot compute suffix of object files: cannot compile

[1] https://git.yoctoproject.org/meta-ti/tree/meta-ti-bsp/conf/machine/include/k3.inc#n6

On Thu Aug 7, 2025 at 1:49 PM CEST, Max Krummenacher via lists.openembedded.org wrote:
> This breaks machines which set `DEFAULTTUNE ?= "aarch64"` and use gcc
> as e.g. done for TI SoC [1].
> For that tune the `TUNE_CCARGS_MARCH` is not set at all resulting in a
> lone `+nocrypto`
> in the compiler arguments. Note that for gcc additionally
> `-march=aarch64` is not an allowed argument.
>
> I don't really know what is a good way forward. Of course one could
> not use aarch64 as a default tune
> to work around the issue.
>
> Any ideas?
>
> Regards
> Max
>
> | checking for aarch64-tdx-linux-gcc... aarch64-tdx-linux-gcc
> +nocrypto -mbranch-protection=standard -fstack-protector-strong  -O2
> -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security
> --sysroot=...aarch64-tdx-linux/libgcc-initial/15.1.0/recipe-sysroot
> | checking for suffix of object files... configure: error: in
> `...aarch64-tdx-linux/libgcc-initial/15.1.0/gcc-15.1.0/build.aarch64-tdx-linux.aarch64-tdx-linux/libgcc':
> | configure: error: cannot compute suffix of object files: cannot compile
>
> [1] https://git.yoctoproject.org/meta-ti/tree/meta-ti-bsp/conf/machine/include/k3.inc#n6

Hi Max,

Can you try this fix commit?

https://lists.openembedded.org/g/openembedded-core/message/221498

It will probably land in master in the coming days.

Best regards,
Mathieu

Hi Mathieu

On Thu, Aug 7, 2025 at 1:58 PM Mathieu Dubois-Briand
<mathieu.dubois-briand@bootlin.com> wrote:
>
> On Thu Aug 7, 2025 at 1:49 PM CEST, Max Krummenacher via lists.openembedded.org wrote:
> > This breaks machines which set `DEFAULTTUNE ?= "aarch64"` and use gcc
> > as e.g. done for TI SoC [1].
> > For that tune the `TUNE_CCARGS_MARCH` is not set at all resulting in a
> > lone `+nocrypto`
> > in the compiler arguments. Note that for gcc additionally
> > `-march=aarch64` is not an allowed argument.
> >
> > I don't really know what is a good way forward. Of course one could
> > not use aarch64 as a default tune
> > to work around the issue.
> >
> > Any ideas?
> >
> > Regards
> > Max
> >
> > | checking for aarch64-tdx-linux-gcc... aarch64-tdx-linux-gcc
> > +nocrypto -mbranch-protection=standard -fstack-protector-strong  -O2
> > -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security
> > --sysroot=...aarch64-tdx-linux/libgcc-initial/15.1.0/recipe-sysroot
> > | checking for suffix of object files... configure: error: in
> > `...aarch64-tdx-linux/libgcc-initial/15.1.0/gcc-15.1.0/build.aarch64-tdx-linux.aarch64-tdx-linux/libgcc':
> > | configure: error: cannot compute suffix of object files: cannot compile
> >
> > [1] https://git.yoctoproject.org/meta-ti/tree/meta-ti-bsp/conf/machine/include/k3.inc#n6
>
> Hi Max,
>
> Can you try this fix commit?
>
> https://lists.openembedded.org/g/openembedded-core/message/221498

This fixes the issue in our setup as well.
Thanks for the pointer and sorry for the noise.

Regards
Max

>
> It will probably land in master in the coming days.
>
> Best regards,
> Mathieu
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>