From patchwork Mon Feb 17 07:56:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 57449 X-Patchwork-Delegate: steve@sakoman.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62F4BC021A4 for ; Mon, 17 Feb 2025 07:56:18 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web11.46431.1739778972856895645 for ; Sun, 16 Feb 2025 23:56:13 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=4143913a06=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 51H5rYTk030500 for ; Mon, 17 Feb 2025 07:56:12 GMT Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [147.11.82.252]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 44thw91nw5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 17 Feb 2025 07:56:11 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.43; Sun, 16 Feb 2025 23:56:10 -0800 Received: from pek-lpg-core5.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.43 via Frontend Transport; Sun, 16 Feb 2025 23:56:10 -0800 From: Hongxu Jia To: Subject: [scarthgap][PATCH 1/2] python3-setuptools-scm: add subprocess timeout control env var Date: Mon, 17 Feb 2025 15:56:08 +0800 Message-ID: <20250217075609.255086-1-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=CZzy5Krl c=1 sm=1 tr=0 ts=67b2eb9b cx=c_pps a=/ZJR302f846pc/tyiSlYyQ==:117 a=/ZJR302f846pc/tyiSlYyQ==:17 a=T2h4t0Lz3GQA:10 a=NEAV23lmAAAA:8 a=t7CeM3EgAAAA:8 a=G7_6LV9d0GMRhNnwkqcA:9 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-GUID: GZg4fel3b9PXEjRtrIM2SI7yWtcx7qhd X-Proofpoint-ORIG-GUID: GZg4fel3b9PXEjRtrIM2SI7yWtcx7qhd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-02-17_03,2025-02-13_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 lowpriorityscore=0 clxscore=1015 priorityscore=1501 malwarescore=0 mlxscore=0 bulkscore=0 phishscore=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.21.0-2501170000 definitions=main-2502170069 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 17 Feb 2025 07:56:18 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/211525 Backport patch from upstream to add subprocess timeout control env var Signed-off-by: Hongxu Jia --- .../0001-fix-957-add-subprocess-timeout.patch | 78 +++++++++++++++++++ .../python/python3-setuptools-scm_8.0.4.bb | 1 + 2 files changed, 79 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch diff --git a/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch b/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch new file mode 100644 index 0000000000..1af1ea0334 --- /dev/null +++ b/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch @@ -0,0 +1,78 @@ +From 2ac4f084def6db53b454b967b062bed2f945aa03 Mon Sep 17 00:00:00 2001 +From: Ronny Pfannschmidt +Date: Fri, 5 Jan 2024 13:34:37 +0100 +Subject: [PATCH] fix #957 - add subprocess timeout + +Upstream-Status: Backport [https://github.com/pypa/setuptools-scm/commit/2ac4f084def6db53b454b967b062bed2f945aa03] +Signed-off-by: Hongxu Jia +--- + changelog.d/20240105_133254_subprocess_timeout_var.md | 4 ++++ + docs/overrides.md | 7 +++++++ + src/setuptools_scm/_run_cmd.py | 11 +++++++++-- + 3 files changed, 20 insertions(+), 2 deletions(-) + create mode 100644 changelog.d/20240105_133254_subprocess_timeout_var.md + +diff --git a/changelog.d/20240105_133254_subprocess_timeout_var.md b/changelog.d/20240105_133254_subprocess_timeout_var.md +new file mode 100644 +index 0000000..78ecab2 +--- /dev/null ++++ b/changelog.d/20240105_133254_subprocess_timeout_var.md +@@ -0,0 +1,4 @@ ++ ++### Changed ++ ++- fix #957 - add subprocess timeout control env var +diff --git a/docs/overrides.md b/docs/overrides.md +index 5114a84..22d285e 100644 +--- a/docs/overrides.md ++++ b/docs/overrides.md +@@ -14,3 +14,10 @@ where the dist name normalization follows adapted PEP 503 semantics. + + setuptools_scm parses the environment variable `SETUPTOOLS_SCM_OVERRIDES_FOR_${NORMALIZED_DIST_NAME}` + as a toml inline map to override the configuration data from `pyproject.toml`. ++ ++## subprocess timeouts ++ ++The environment variable `SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT` allows to override the subprocess timeout. ++The default is 40 seconds and should work for most needs. However, users with git lfs + windows reported ++situations where this was not enough. ++ +diff --git a/src/setuptools_scm/_run_cmd.py b/src/setuptools_scm/_run_cmd.py +index 3dfbda5..a64f904 100644 +--- a/src/setuptools_scm/_run_cmd.py ++++ b/src/setuptools_scm/_run_cmd.py +@@ -25,7 +25,12 @@ else: + # unfortunately github CI for windows sometimes needs + # up to 30 seconds to start a command + +-BROKEN_TIMEOUT: Final[int] = 40 ++ ++def _get_timeout(env: Mapping[str, str]) -> int: ++ return int(env.get("SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT") or 40) ++ ++ ++BROKEN_TIMEOUT: Final[int] = _get_timeout(os.environ) + + log = _log.log.getChild("run_cmd") + +@@ -132,7 +137,7 @@ def run( + *, + strip: bool = True, + trace: bool = True, +- timeout: int = BROKEN_TIMEOUT, ++ timeout: int | None = None, + check: bool = False, + ) -> CompletedProcess: + if isinstance(cmd, str): +@@ -141,6 +146,8 @@ def run( + cmd = [os.fspath(x) for x in cmd] + cmd_4_trace = " ".join(map(_unsafe_quote_for_display, cmd)) + log.debug("at %s\n $ %s ", cwd, cmd_4_trace) ++ if timeout is None: ++ timeout = BROKEN_TIMEOUT + res = subprocess.run( + cmd, + capture_output=True, +-- +2.25.1 + diff --git a/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb b/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb index 67ac407c38..62a15f9d7f 100644 --- a/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb +++ b/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb @@ -12,6 +12,7 @@ inherit pypi python_setuptools_build_meta SRC_URI += " \ file://0001-fix-timeout-while-using-big-git-repo.patch \ + file://0001-fix-957-add-subprocess-timeout.patch \ " UPSTREAM_CHECK_REGEX = "scm-(?P.*)\.tar" From patchwork Mon Feb 17 07:56:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hongxu Jia X-Patchwork-Id: 57450 X-Patchwork-Delegate: steve@sakoman.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63F64C021A0 for ; Mon, 17 Feb 2025 07:56:28 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.web10.46631.1739778973507074802 for ; Sun, 16 Feb 2025 23:56:13 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=4143913a06=hongxu.jia@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 51H5rYTl030500 for ; Mon, 17 Feb 2025 07:56:12 GMT Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [147.11.82.252]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 44thw91nw5-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 17 Feb 2025 07:56:12 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (147.11.82.252) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.43; Sun, 16 Feb 2025 23:56:11 -0800 Received: from pek-lpg-core5.wrs.com (147.11.136.210) by ala-exchng01.corp.ad.wrs.com (147.11.82.252) with Microsoft SMTP Server id 15.1.2507.43 via Frontend Transport; Sun, 16 Feb 2025 23:56:11 -0800 From: Hongxu Jia To: Subject: [scarthgap][PATCH 2/2] setuptools3-base.bbclass: override default subprocess timeout Date: Mon, 17 Feb 2025 15:56:09 +0800 Message-ID: <20250217075609.255086-2-hongxu.jia@windriver.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250217075609.255086-1-hongxu.jia@windriver.com> References: <20250217075609.255086-1-hongxu.jia@windriver.com> MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=CZzy5Krl c=1 sm=1 tr=0 ts=67b2eb9c cx=c_pps a=/ZJR302f846pc/tyiSlYyQ==:117 a=/ZJR302f846pc/tyiSlYyQ==:17 a=IkcTkHD0fZMA:10 a=T2h4t0Lz3GQA:10 a=NEAV23lmAAAA:8 a=t7CeM3EgAAAA:8 a=zwXmAwp_x2hHYwLl0w4A:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-GUID: 19FDPpHIpEL3CRdoZWitu5gNXHSZi1oD X-Proofpoint-ORIG-GUID: 19FDPpHIpEL3CRdoZWitu5gNXHSZi1oD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-02-17_03,2025-02-13_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 lowpriorityscore=0 clxscore=1015 priorityscore=1501 malwarescore=0 mlxscore=0 bulkscore=0 phishscore=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.21.0-2501170000 definitions=main-2502170069 X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-0064b401.pphosted.com id 51H5rYTl030500 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 17 Feb 2025 07:56:28 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/211526 The environment variable SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT allows to override the subprocess timeout. The default is 40 seconds and should work for most needs.[1] However, it was not enough while using git shallow tarball and starting multiple Yocto world builds in one host. |   File "tmp/work/x86_64-linux/python3-scancode-native/32.1.0/recipe-sysroot- native/usr/lib/python3.13/subprocess.py", line 1263, in _check_timeout |     raise TimeoutExpired( |     ...<2 lines>... |             stderr=b''.join(stderr_seq) if stderr_seq else None) | subprocess.TimeoutExpired: Command '['git', '--git-dir', 'tmp/work/x86_64- linux/python3-scancode-native/32.1.0/git/.git', 'status', '--porcelain', '--untracked-files=no']' timed out after 40 seconds Explicitly set variable SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT to 600s in bbclass, and we could override it in local.conf [1] https://github.com/pypa/setuptools-scm/blob/main/docs/overrides.md Signed-off-by: Hongxu Jia --- meta/classes-recipe/setuptools3-base.bbclass | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/meta/classes-recipe/setuptools3-base.bbclass b/meta/classes-recipe/setuptools3-base.bbclass index 27af6abc58..190d9e6e3a 100644 --- a/meta/classes-recipe/setuptools3-base.bbclass +++ b/meta/classes-recipe/setuptools3-base.bbclass @@ -23,6 +23,10 @@ export CCSHARED = "-fPIC -DPIC" # the python executable export LINKFORSHARED = "${SECURITY_CFLAGS} -Xlinker -export-dynamic" +# The environment variable SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT allows +# to override the subprocess timeout. +export SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT ??= "600" + FILES:${PN} += "${PYTHON_SITEPACKAGES_DIR}" FILES:${PN}-staticdev += "${PYTHON_SITEPACKAGES_DIR}/*.a" FILES:${PN}-dev += "${PYTHON_SITEPACKAGES_DIR}/*.la"