| Message ID | 20250331133044.1944108-1-richard.purdie@linuxfoundation.org |
|---|---|
| State | Accepted, archived |
| Commit | 2b6a6d4199e4eed65f20351d077a04a7cf9da460 |
| Headers | show |
| Series | rust-llvm: Compile llvm to use dynamic libraries | expand |
On Mon, Mar 31, 2025 at 6:30 AM Richard Purdie via lists.openembedded.org <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > Our main llvm recipe uses dynamic linking already but rust-llvm does not. > Enabling this significanly reduces the size of llvm-rust to about a third > of what it was, which is a bettter configuration for us leading to smaller > TMPDIR and sstate objects It will slow down the compiler execution and sometimes its could be significant as well we have seen this with clang so it would be good to have some idea around that least we provide a slow compiler compared to other distro infrastructures > > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > meta/recipes-devtools/rust/rust-llvm_1.85.1.bb | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > b/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > index 6a4d2957288..566348c617d 100644 > --- a/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > +++ b/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > @@ -44,6 +44,8 @@ EXTRA_OECMAKE = " \ > -DLLVM_ENABLE_FFI=OFF \ > -DLLVM_INSTALL_UTILS=ON \ > -DLLVM_BUILD_EXAMPLES=OFF \ > + -DLLVM_BUILD_LLVM_DYLIB=ON \ > + -DLLVM_LINK_LLVM_DYLIB=ON \ > -DLLVM_INCLUDE_EXAMPLES=OFF \ > -DLLVM_BUILD_TESTS=OFF \ > -DLLVM_INCLUDE_TESTS=OFF \ > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#213979): > https://lists.openembedded.org/g/openembedded-core/message/213979 > Mute This Topic: https://lists.openembedded.org/mt/112003864/1997914 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On 2025-03-31 12:13 p.m., Khem Raj via lists.openembedded.org wrote: > > > On Mon, Mar 31, 2025 at 6:30 AM Richard Purdie via > lists.openembedded.org <http://lists.openembedded.org> > <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > > Our main llvm recipe uses dynamic linking already but rust-llvm > does not. > Enabling this significanly reduces the size of llvm-rust to about > a third > of what it was, which is a bettter configuration for us leading to > smaller > TMPDIR and sstate objects > > > It will slow down the compiler execution and sometimes its could be > significant as well we have seen this with clang so it would be good > to have some idea around that least we provide a slow compiler > compared to other distro infrastructures We'll get some performance info from the YP AB perf builds. Sundeep, Once we see if there's a noticeable difference in those YP AB runs, we should look into bench-marking the build time for rust and check for any difference in building Rust in the SDK and maybe even on target as another benchmark. This is less urgent than getting the failing and skipped rust tests enabled but it shouldn't wait too long. ../Randy > > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > <http://rust-llvm_1.85.1.bb> | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > <http://rust-llvm_1.85.1.bb> > b/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > <http://rust-llvm_1.85.1.bb> > index 6a4d2957288..566348c617d 100644 > --- a/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > <http://rust-llvm_1.85.1.bb> > +++ b/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb > <http://rust-llvm_1.85.1.bb> > @@ -44,6 +44,8 @@ EXTRA_OECMAKE = " \ > -DLLVM_ENABLE_FFI=OFF \ > -DLLVM_INSTALL_UTILS=ON \ > -DLLVM_BUILD_EXAMPLES=OFF \ > + -DLLVM_BUILD_LLVM_DYLIB=ON \ > + -DLLVM_LINK_LLVM_DYLIB=ON \ > -DLLVM_INCLUDE_EXAMPLES=OFF \ > -DLLVM_BUILD_TESTS=OFF \ > -DLLVM_INCLUDE_TESTS=OFF \ > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#213987):https://lists.openembedded.org/g/openembedded-core/message/213987 > Mute This Topic:https://lists.openembedded.org/mt/112003864/3616765 > Group Owner:openembedded-core+owner@lists.openembedded.org > Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Mon, 2025-03-31 at 15:08 -0400, Randy MacLeod wrote: > > On 2025-03-31 12:13 p.m., Khem Raj via lists.openembedded.org wrote: > > On Mon, Mar 31, 2025 at 6:30 AM Richard Purdie via > > lists.openembedded.org > > <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > > > Our main llvm recipe uses dynamic linking already but rust-llvm > > > does not. > > > Enabling this significanly reduces the size of llvm-rust to about > > > a third > > > of what it was, which is a bettter configuration for us leading > > > to smaller > > > TMPDIR and sstate objects > > > > > > > > > > > > > It will slow down the compiler execution and sometimes its could be > > significant as well we have seen this with clang so it would be > > good to have some idea around that least we provide a slow compiler > > compared to other distro infrastructures > > > > > > > > We'll get some performance info from the YP AB perf builds. https://valkyrie.yocto.io/pub/non-release/20250331-105/testresults/buildperf-debian11/perf-debian12-vk_master-next_20250331181527_9ba5b4f4e4.html https://valkyrie.yocto.io/pub/non-release/20250331-104/testresults/buildperf-alma8/perf-alma8-vk_master-next_20250331181513_9ba5b4f4e4.html The build time is fractionally faster (1%?) but hard to say within noise. The diskspace drop is convincing, 2.3GB less TMPDIR size (2.3%). The rm_work TMPDIR size also drops by 1GB (6%) which is particularly significant and nice to have. In builds with rust-llvm as well as rust-llvm-native, this will double up for a double win. Cheers, Richard
On Mon, Mar 31, 2025 at 1:44 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Mon, 2025-03-31 at 15:08 -0400, Randy MacLeod wrote: > > On 2025-03-31 12:13 p.m., Khem Raj via lists.openembedded.org wrote: > > On Mon, Mar 31, 2025 at 6:30 AM Richard Purdie via lists.openembedded.org <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > > Our main llvm recipe uses dynamic linking already but rust-llvm does not. > Enabling this significanly reduces the size of llvm-rust to about a third > of what it was, which is a bettter configuration for us leading to smaller > TMPDIR and sstate objects > > > It will slow down the compiler execution and sometimes its could be significant as well we have seen this with clang so it would be good to have some idea around that least we provide a slow compiler compared to other distro infrastructures > > We'll get some performance info from the YP AB perf builds. > > > https://valkyrie.yocto.io/pub/non-release/20250331-105/testresults/buildperf-debian11/perf-debian12-vk_master-next_20250331181527_9ba5b4f4e4.html > https://valkyrie.yocto.io/pub/non-release/20250331-104/testresults/buildperf-alma8/perf-alma8-vk_master-next_20250331181513_9ba5b4f4e4.html > > The build time is fractionally faster (1%?) but hard to say within noise. That's encouraging. I don't know if we have some large rust app which we compile using rust compiler with and without this patch. There is uutils in meta-oe and python3-pydantic-core in meta-python which can show just the rust compiler's impact. > > The diskspace drop is convincing, 2.3GB less TMPDIR size (2.3%). > > The rm_work TMPDIR size also drops by 1GB (6%) which is particularly significant and nice to have. > > In builds with rust-llvm as well as rust-llvm-native, this will double up for a double win. > > Cheers, > > Richard
On Mon, 2025-03-31 at 21:44 +0100, Richard Purdie via lists.openembedded.org wrote: > On Mon, 2025-03-31 at 15:08 -0400, Randy MacLeod wrote: > > > > On 2025-03-31 12:13 p.m., Khem Raj via lists.openembedded.org > > wrote: > > > > > On Mon, Mar 31, 2025 at 6:30 AM Richard Purdie via > > > lists.openembedded.org > > > <richard.purdie=linuxfoundation.org@lists.openembedded.org> > > > wrote: > > > > Our main llvm recipe uses dynamic linking already but rust-llvm > > > > does not. > > > > Enabling this significanly reduces the size of llvm-rust to > > > > about a third > > > > of what it was, which is a bettter configuration for us > > > > leading to smaller > > > > TMPDIR and sstate objects > > > > > > > > > > > > > > > > > > > It will slow down the compiler execution and sometimes its could > > > be significant as well we have seen this with clang so it would > > > be good to have some idea around that least we provide a slow > > > compiler compared to other distro infrastructures > > > > > > > > > > > > > > > > We'll get some performance info from the YP AB perf builds. > > https://valkyrie.yocto.io/pub/non-release/20250331-105/testresults/buildperf-debian11/perf-debian12-vk_master-next_20250331181527_9ba5b4f4e4.html > https://valkyrie.yocto.io/pub/non-release/20250331-104/testresults/buildperf-alma8/perf-alma8-vk_master-next_20250331181513_9ba5b4f4e4.html > > The build time is fractionally faster (1%?) but hard to say within > noise. > > The diskspace drop is convincing, 2.3GB less TMPDIR size (2.3%). > > The rm_work TMPDIR size also drops by 1GB (6%) which is particularly > significant and nice to have. > > In builds with rust-llvm as well as rust-llvm-native, this will > double up for a double win. The analysis is a bit flawed as it is contrasting master with master- next which has 1.85 as well as other changes. We need to remove that from the equation. Cheers, Richard
On Mon, 2025-03-31 at 22:49 +0100, Richard Purdie via lists.openembedded.org wrote: > On Mon, 2025-03-31 at 21:44 +0100, Richard Purdie via > lists.openembedded.org wrote: > > On Mon, 2025-03-31 at 15:08 -0400, Randy MacLeod wrote: > > > > > > On 2025-03-31 12:13 p.m., Khem Raj via lists.openembedded.org > > > wrote: > > > > > > > On Mon, Mar 31, 2025 at 6:30 AM Richard Purdie via > > > > lists.openembedded.org > > > > <richard.purdie=linuxfoundation.org@lists.openembedded.org> > > > > wrote: > > > > > Our main llvm recipe uses dynamic linking already but rust- > > > > > llvm > > > > > does not. > > > > > Enabling this significanly reduces the size of llvm-rust to > > > > > about a third > > > > > of what it was, which is a bettter configuration for us > > > > > leading to smaller > > > > > TMPDIR and sstate objects > > > > > > > > > > > > > > > > > > > > > > > > > It will slow down the compiler execution and sometimes its > > > > could > > > > be significant as well we have seen this with clang so it would > > > > be good to have some idea around that least we provide a slow > > > > compiler compared to other distro infrastructures > > > > > > > > > > > > > > > > > > > > > > We'll get some performance info from the YP AB perf builds. > > > > https://valkyrie.yocto.io/pub/non-release/20250331-105/testresults/buildperf-debian11/perf-debian12-vk_master-next_20250331181527_9ba5b4f4e4.html > > https://valkyrie.yocto.io/pub/non-release/20250331-104/testresults/buildperf-alma8/perf-alma8-vk_master-next_20250331181513_9ba5b4f4e4.html > > > > The build time is fractionally faster (1%?) but hard to say within > > noise. > > > > The diskspace drop is convincing, 2.3GB less TMPDIR size (2.3%). > > > > The rm_work TMPDIR size also drops by 1GB (6%) which is > > particularly > > significant and nice to have. > > > > In builds with rust-llvm as well as rust-llvm-native, this will > > double up for a double win. > > The analysis is a bit flawed as it is contrasting master with master- > next which has 1.85 as well as other changes. We need to remove that > from the equation. There are a direct comparison with just the patch added: https://valkyrie.yocto.io/pub/non-release/20250331-129/testresults/buildperf-debian11/perf-debian12-vk_master-next_20250331220727_26fa862acd.html https://valkyrie.yocto.io/pub/non-release/20250331-128/testresults/buildperf-alma8/perf-alma8-vk_master-next_20250331220715_26fa862acd.html So the buildtime is around the same (or less with rm_work), it is hard to say from the daya. TMPDIR is 3.6GB less and 1GB less with rm_work so definitely improved. Enough of an improvement in space usage I think this is worthwhile. In general space usage does correlate with build speed. Cheers, Richard
On Tue, Apr 1, 2025 at 12:35 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Mon, 2025-03-31 at 22:49 +0100, Richard Purdie via > lists.openembedded.org wrote: > > On Mon, 2025-03-31 at 21:44 +0100, Richard Purdie via > > lists.openembedded.org wrote: > > > On Mon, 2025-03-31 at 15:08 -0400, Randy MacLeod wrote: > > > > > > > > On 2025-03-31 12:13 p.m., Khem Raj via lists.openembedded.org > > > > wrote: > > > > > > > > > On Mon, Mar 31, 2025 at 6:30 AM Richard Purdie via > > > > > lists.openembedded.org > > > > > <richard.purdie=linuxfoundation.org@lists.openembedded.org> > > > > > wrote: > > > > > > Our main llvm recipe uses dynamic linking already but rust- > > > > > > llvm > > > > > > does not. > > > > > > Enabling this significanly reduces the size of llvm-rust to > > > > > > about a third > > > > > > of what it was, which is a bettter configuration for us > > > > > > leading to smaller > > > > > > TMPDIR and sstate objects > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will slow down the compiler execution and sometimes its > > > > > could > > > > > be significant as well we have seen this with clang so it would > > > > > be good to have some idea around that least we provide a slow > > > > > compiler compared to other distro infrastructures > > > > > > > > > > > > > > > > > > > > > > > > > > > > We'll get some performance info from the YP AB perf builds. > > > > > > https://valkyrie.yocto.io/pub/non-release/20250331-105/testresults/buildperf-debian11/perf-debian12-vk_master-next_20250331181527_9ba5b4f4e4.html > > > https://valkyrie.yocto.io/pub/non-release/20250331-104/testresults/buildperf-alma8/perf-alma8-vk_master-next_20250331181513_9ba5b4f4e4.html > > > > > > The build time is fractionally faster (1%?) but hard to say within > > > noise. > > > > > > The diskspace drop is convincing, 2.3GB less TMPDIR size (2.3%). > > > > > > The rm_work TMPDIR size also drops by 1GB (6%) which is > > > particularly > > > significant and nice to have. > > > > > > In builds with rust-llvm as well as rust-llvm-native, this will > > > double up for a double win. > > > > The analysis is a bit flawed as it is contrasting master with master- > > next which has 1.85 as well as other changes. We need to remove that > > from the equation. > > There are a direct comparison with just the patch added: > > https://valkyrie.yocto.io/pub/non-release/20250331-129/testresults/buildperf-debian11/perf-debian12-vk_master-next_20250331220727_26fa862acd.html > https://valkyrie.yocto.io/pub/non-release/20250331-128/testresults/buildperf-alma8/perf-alma8-vk_master-next_20250331220715_26fa862acd.html > > So the buildtime is around the same (or less with rm_work), it is hard > to say from the daya. TMPDIR is 3.6GB less and 1GB less with rm_work so > definitely improved. > > Enough of an improvement in space usage I think this is worthwhile. In > general space usage does correlate with build speed. > Thanks for sharing these numbers, I am ok to apply this as well. > Cheers, > > Richard
diff --git a/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb b/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb index 6a4d2957288..566348c617d 100644 --- a/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb +++ b/meta/recipes-devtools/rust/rust-llvm_1.85.1.bb @@ -44,6 +44,8 @@ EXTRA_OECMAKE = " \ -DLLVM_ENABLE_FFI=OFF \ -DLLVM_INSTALL_UTILS=ON \ -DLLVM_BUILD_EXAMPLES=OFF \ + -DLLVM_BUILD_LLVM_DYLIB=ON \ + -DLLVM_LINK_LLVM_DYLIB=ON \ -DLLVM_INCLUDE_EXAMPLES=OFF \ -DLLVM_BUILD_TESTS=OFF \ -DLLVM_INCLUDE_TESTS=OFF \
Our main llvm recipe uses dynamic linking already but rust-llvm does not. Enabling this significanly reduces the size of llvm-rust to about a third of what it was, which is a bettter configuration for us leading to smaller TMPDIR and sstate objects. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- meta/recipes-devtools/rust/rust-llvm_1.85.1.bb | 2 ++ 1 file changed, 2 insertions(+)