diff mbox series

[meta-python] python3-grpcio(-tools): fix build concurrency issue

Message ID 20250228205710.1739487-1-peter.marko@siemens.com
State Accepted
Headers show
Series [meta-python] python3-grpcio(-tools): fix build concurrency issue | expand

Commit Message

Peter Marko Feb. 28, 2025, 8:57 p.m. UTC 
From: Peter Marko <peter.marko@siemens.com>

Set GRPC_PYTHON_BUILD_EXT_COMPILER_JOBS to limit spawned compiler
processes. Without this it uses all available CPUs (via
multiprocessing.cpu_count()) and can exhaust build host since there are
lot of files to compile (e.g. with 128 cores it manages to spawn 128 gcc
processes)

Note that this is a general problem for all setuptools based builds with
build_ext compilation which can either compile with 1 thread or
cpu_count threads. grpcio hot-patches setuptools and allows to set
specific build concurrency value.

Signed-off-by: Peter Marko <peter.marko@siemens.com>
---
 .../recipes-devtools/python/python3-grpcio-tools_1.70.0.bb    | 4 ++++
 meta-python/recipes-devtools/python/python3-grpcio_1.70.0.bb  | 4 ++++
 2 files changed, 8 insertions(+)
diff mbox series

Patch

diff --git a/meta-python/recipes-devtools/python/python3-grpcio-tools_1.70.0.bb b/meta-python/recipes-devtools/python/python3-grpcio-tools_1.70.0.bb
index 8af6bb5714..38440edbee 100644
--- a/meta-python/recipes-devtools/python/python3-grpcio-tools_1.70.0.bb
+++ b/meta-python/recipes-devtools/python/python3-grpcio-tools_1.70.0.bb
@@ -19,6 +19,10 @@  SRC_URI[sha256sum] = "e578fee7c1c213c8e471750d92631d00f178a15479fb2cb3b939a07fc1
 
 RDEPENDS:${PN} = "python3-grpcio"
 
+do_compile:prepend() {
+    export GRPC_PYTHON_BUILD_EXT_COMPILER_JOBS="${@oe.utils.parallel_make(d, False)}"
+}
+
 BBCLASSEXTEND = "native nativesdk"
 
 CVE_PRODUCT += "grpc:grpc"
diff --git a/meta-python/recipes-devtools/python/python3-grpcio_1.70.0.bb b/meta-python/recipes-devtools/python/python3-grpcio_1.70.0.bb
index 5af05ca2c9..b858a3b050 100644
--- a/meta-python/recipes-devtools/python/python3-grpcio_1.70.0.bb
+++ b/meta-python/recipes-devtools/python/python3-grpcio_1.70.0.bb
@@ -29,6 +29,10 @@  export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL = "1"
 export GRPC_PYTHON_BUILD_SYSTEM_RE2 = "1"
 export GRPC_PYTHON_BUILD_SYSTEM_ZLIB = "1"
 
+do_compile:prepend() {
+    export GRPC_PYTHON_BUILD_EXT_COMPILER_JOBS="${@oe.utils.parallel_make(d, False)}"
+}
+
 GRPC_CFLAGS ?= ""
 GRPC_CFLAGS:append:toolchain-clang = " -fvisibility=hidden -fno-wrapv -fno-exceptions"
 export GRPC_PYTHON_CFLAGS = "${GRPC_CFLAGS}"