From patchwork Wed Apr 1 13:45:32 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: naftaly.ralamboarivony@smile.fr X-Patchwork-Id: 84971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7ABA1107637B for ; Wed, 1 Apr 2026 13:46:00 +0000 (UTC) Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.13472.1775051150548878907 for ; Wed, 01 Apr 2026 06:45:50 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=2iGJa+MN; spf=pass (domain: smile.fr, ip: 209.85.221.47, mailfrom: naftaly.ralamboarivony@smile.fr) Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-43cfde3c3f3so2789071f8f.3 for ; Wed, 01 Apr 2026 06:45:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1775051149; x=1775655949; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=CSzW0DaoKq5WeZxCuLD3vQNRGp67csGHczTV6p3B08A=; b=2iGJa+MNy1C4PrNb4JaMiok6+1RppgElZ1S3oy+EF9ca22lOWLzTi1acyw63KVSX+y rTOS0+wqIySrjTmEaaG9c5RsxuGEYCviBasDZd1VAR1GwD2GhJd0ZQQagFTyzH1mEUwb JrL0fOkjfl2zALNN+4Rr7BiTGIHry98xiRK8A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775051149; x=1775655949; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=CSzW0DaoKq5WeZxCuLD3vQNRGp67csGHczTV6p3B08A=; b=m1ul9vPERrf+nsbDt1W74evXp34qnbEhUicDcS6hplU8UNQCwaflRhQPwoqHmgtgn4 XR5MpZ9sonXMWUpf/nA5JTsXqvnMPMw/An3k/cvvjikFYvPTeSCEoM4fPnCXXGBO1SK9 moIQ9qewvpdcankR0B4gSOJd7WYSHDm8h0FVtXx6xZbo7hnErmoV1CqJOub3ncN7aZoV pVEVTQzZA3CkZ70dVE0chgrJUj6unA9TFsP8zWgxI3zjn1HGXHnwyPHBh2AS3/tm4HKQ A/R1gy37xrYfqrQcMP4rq32q7hmbnYtaEYfwhO8DYpQWPDCXTq5nzgMuXh+oinYGAzYa 83Og== X-Gm-Message-State: AOJu0Yyd4bk2uzZNrVPhRp0bMaSbCAVRwVk2cDOzGhJlG+eOE1K7T8wK 4PlW/IlYA7bS3t5cINSAFP1W18uPvEt3rgL4I3o11eAv5Scou4eNE43dN7WY6kYhwXi6w7V6/L9 s+oR5vOA= X-Gm-Gg: ATEYQzyO44WEbz09ThjerGVeqf0tPvm/XRuTVikmUgyjLEmdCHfPoVxirmIqPiO5rIm +RihN522xbrJJFx3rLAfd7Ct3JAdpPQHXldeJt9xAxJTYR4epQdlRm5694XKvffNn9Z6sul2fh2 v3ULjAtZ7DOiFyOT0STO35CKEKMNwGyCIaJAkZIOCj/P7Ne8nWo9uh9YtxYOOl2Mc80RV05RMir C9cQL82MYExJV7oA8+t285VZ7EJwBRKoEa3v3TEGRaUnDZqLQN13Iufc/uPb41CfAHyfIq6VHUV hEwzEQE+yN+Yzz3mJZgIfIlR07+amWHfvTZdXONRic1cbnY1Bs4Nf8IJ/bvIxed6ddD3N+LK1IU qnWyXwgAhfibsw4GUb1gN/Q/cHxo8G0Ae3e1NaIt1lVQPBIKiLHnrLPnmii5DszqSisT+fYsX20 HL3EWv29JDSn9sEd0ySPemxR1mlxWP622/mD29MeZ8F7E3P0YzZLc0brtYh9s3SwwWUj6TzGSMq l9KQqpqXf08uHlUyvjgodK20L2exDQvH3+0O842DKPSMQ== X-Received: by 2002:adf:e60d:0:b0:43d:1e1f:9440 with SMTP id ffacd0b85a97d-43d1e1f95a5mr91248f8f.48.1775051148678; Wed, 01 Apr 2026 06:45:48 -0700 (PDT) Received: from P-ASN-DERVAUX.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43d1e1fe0b0sm84653f8f.0.2026.04.01.06.45.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Apr 2026 06:45:47 -0700 (PDT) From: naftaly.ralamboarivony@smile.fr To: openembedded-core@lists.openembedded.org Cc: Naftaly RALAMBOARIVONY Subject: [OE-core][PATCH v2 1/2] patchtest: return non-zero exit code on test failure Date: Wed, 1 Apr 2026 15:45:32 +0200 Message-Id: <20260401134533.322260-1-naftaly.ralamboarivony@smile.fr> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 01 Apr 2026 13:46:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/234348 From: Naftaly RALAMBOARIVONY Update patchtest to return a non-zero exit code when a test fails instead of always exiting successfully. This enables proper failure detection in selftests and CI pipelines relying on the command return status. Signed-off-by: Naftaly RALAMBOARIVONY --- changes in v2: - Removed the --error-on-failure option; if a test fails, a non-zero value is returned --- scripts/patchtest | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/scripts/patchtest b/scripts/patchtest index 9218db232a..143cf08572 100755 --- a/scripts/patchtest +++ b/scripts/patchtest @@ -183,6 +183,7 @@ def print_result_message(preresult, postresult): print("----------------------------------------------------------------------\n") def main(): + ret = 0 tmp_patch = False patch_path = PatchtestParser.patch_path log_results = PatchtestParser.log_results @@ -214,13 +215,15 @@ def main(): try: if log_path: - run(patch, log_path) + ret = run(patch, log_path) else: - run(patch) + ret = run(patch) finally: if tmp_patch: os.remove(patch) + return ret + if __name__ == '__main__': ret = 1 From patchwork Wed Apr 1 13:45:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: naftaly.ralamboarivony@smile.fr X-Patchwork-Id: 84972 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D5DF107637D for ; Wed, 1 Apr 2026 13:46:00 +0000 (UTC) Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.13602.1775051151614721238 for ; Wed, 01 Apr 2026 06:45:51 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=Ormh5Eps; spf=pass (domain: smile.fr, ip: 209.85.221.53, mailfrom: naftaly.ralamboarivony@smile.fr) Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-43d0deb7ad5so1614217f8f.2 for ; Wed, 01 Apr 2026 06:45:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1775051150; x=1775655950; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dO3aftoLWTrbD9DcHccffKt4CJxGA/juM1W5MFbuL2w=; b=Ormh5Epsf2Glj+lbzUEiaSnNWKx08p4tKGEixWDWhJNloVUb7XrYuIIhcxxfz0MeKf nq7uvjkMi0dNe+8qhsR68k9QCfGT4C98cALTbTqTD0v7mCVbDoU/db24phUoLMW6Nv9G UpO4qVaxfAuqRCQ3rnaIro1qggFpkJBD8nxPo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775051150; x=1775655950; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=dO3aftoLWTrbD9DcHccffKt4CJxGA/juM1W5MFbuL2w=; b=E2EK2NH3gH3OfZYTCXhgxvKzPFuW7Y/WTB3vc1BD5kq45yOzolRNFWiqA8Sx0FDhXB bVObaM3l1tpfjlhgNfsDu/BU4rU7ixOmYTOgJ/rCHdOQj3dvV/ncSaAxL6ca9eNOD01t UdLWWWm/o9lPrZmVz0PDa963BUyKxysmoS10WtkYNJdS4fSaBjjUUlvNYKXSO7x/8Xyl 1GsuylRh+ng5GaEhfbHnF199vElmjjqGW19rj6gVAlptFZOKlf8I60oYPJ/i1YpaihJv 99DvTAsO4ykWc9OHLbuzn5tsnJRyOZ5qYZWCVIh1qU1AAlALDKlU1AGIYIsqitOJx5To TLGQ== X-Gm-Message-State: AOJu0YzR0Hd82eEL7sfWZhGMAp1BMfzlQg5FTNBkLJfQb8ICknk+cuOb j7iY96V8/17bdszgTuMYr+U8ywyHAHhcJyHWBzfqerfEC9gGEIEa0w1sqLybzqwvLc304kvlHjB M1VtmVVc= X-Gm-Gg: ATEYQzwZ+msehdFmtqx5AQTCwBG/reHFXnkz8vitUTo0JGdTJuv2V63cojQZe3p9Mj9 0f0HFX/qEGdkRHwDC/utTlGsrwOmmzI5vOy+ZWPNK+ikj6uH/m1c4PkFfNfKpE0lkaDe1AbYtfK 6ve/T2c8XcJocIymdcxPQO7M49sq2VD9ZKn2imnqAOhsNEHf3SCvh0TJrUqpyokF9r7PCb63vuu FsqKGziDJupqUBRfi6VuSQll6Q7IWSqyy42JwWXZyXgBnp6HK22RCnhulcDUw8f8XAf/A/b1+Ea vEz4c9MsPS6oTVuy8UWHNe3igXWcBareAVQGvGFWTJ4Fz9LFmYgKIZsTfXVP6aG025S1V6E6d5W SLRFXrnye0i6vNyLKelPBAAiwFq5oouuzunFB9qK+O9PxruN1cCAVqgf+z29pP7QT1nuTbE5bHp yBF8xEgMELh4yfsIqfx+e2l0OP0m8uTvn4YpNtpo7kf2gVSdcS2NbKcPOCtZ/MjMjINYuTeJFQM 5QgcZJ9Yft+GJHsFMB9vd2HBhieUAiwL9WvE4nAp/B8Wg== X-Received: by 2002:a05:6000:2303:b0:439:beb9:5a96 with SMTP id ffacd0b85a97d-43d150e8ba7mr7043603f8f.31.1775051149713; Wed, 01 Apr 2026 06:45:49 -0700 (PDT) Received: from P-ASN-DERVAUX.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43d1e1fe0b0sm84653f8f.0.2026.04.01.06.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Apr 2026 06:45:49 -0700 (PDT) From: naftaly.ralamboarivony@smile.fr To: openembedded-core@lists.openembedded.org Cc: Naftaly RALAMBOARIVONY Subject: [OE-core][PATCH v2 2/2] patchtest/selftest: Update selftest to handle non-zero exit code Date: Wed, 1 Apr 2026 15:45:33 +0200 Message-Id: <20260401134533.322260-2-naftaly.ralamboarivony@smile.fr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260401134533.322260-1-naftaly.ralamboarivony@smile.fr> References: <20260401134533.322260-1-naftaly.ralamboarivony@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 01 Apr 2026 13:46:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/234349 From: Naftaly RALAMBOARIVONY Use subprocess.run() instead of subprocess.check_output(), supbrocess.run() return CompletedProcess instance who provide: - returncode - stdout and stderr with stderr=subprocess.STDOUT argument Update callers of test() to capture the return code returned by patchtest along with the command output. Include the command return code in analyze_result() and use it to validate test for XPASS, XFAIL and XSKIP. This allows selftest to take the command exit status into account when analyzing results. Signed-off-by: Naftaly RALAMBOARIVONY --- changes in v2: - Removed the --error-on-failure option; if a test fails, a non-zero value is returned. - Updated selftest to align with the new patchtest behavior when a test fails --- meta/lib/patchtest/selftest/selftest | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/meta/lib/patchtest/selftest/selftest b/meta/lib/patchtest/selftest/selftest index 43cccf4c85..26d1b61227 100755 --- a/meta/lib/patchtest/selftest/selftest +++ b/meta/lib/patchtest/selftest/selftest @@ -51,20 +51,20 @@ def get_patches(patchesdir): }) return patch_list -def analyze_result(results, patch, counts): +def analyze_result(results, patch, counts, return_code): testid = patch["testid"] expected_result = str(patch["expected"]) for resultline in results.splitlines(): if testid in resultline: result, _ = resultline.split(':', 1) - if expected_result.upper() == "FAIL" and result.upper() == "FAIL": + if expected_result.upper() == "FAIL" and result.upper() == "FAIL" and return_code != 0: counts["xfail"] = counts["xfail"] + 1 print("XFAIL: %s (file: %s)" % (testid.strip("."), os.path.basename(patch["patch"]))) - elif expected_result.upper() == "PASS" and result.upper() == "PASS": + elif expected_result.upper() == "PASS" and result.upper() == "PASS" and return_code == 0: counts["xpass"] = counts["xpass"] + 1 print("XPASS: %s (file: %s)" % (testid.strip("."), os.path.basename(patch["patch"]))) - elif expected_result.upper() == "SKIP" and result.upper() == "SKIP": + elif expected_result.upper() == "SKIP" and result.upper() == "SKIP" and return_code == 0: counts["xskip"] = counts["xskip"] + 1 print("XSKIP: %s (file: %s)" % (testid.strip("."), os.path.basename(patch["patch"]))) else: @@ -135,17 +135,17 @@ def test(root, patch): patchpath = os.path.abspath(os.path.join(root, patch)) cmd = 'patchtest --base-commit HEAD --repodir %s --testdir %s/tests --patch %s' % (repodir, topdir, patchpath) - results = subprocess.check_output(cmd, stderr=subprocess.STDOUT, universal_newlines=True, shell=True) + results = subprocess.run(cmd, capture_output=True, universal_newlines=True, shell=True) - return results + return results.returncode, results.stdout def test_head_attached(patches, counts, branch): git_attach_head(branch) git_state_before = get_git_state() for patch_info in patches: - results = test(patch_info["root"], patch_info["patch"]) - counts = analyze_result(results, patch_info, counts) + return_code, results = test(patch_info["root"], patch_info["patch"]) + counts = analyze_result(results, patch_info, counts, return_code) git_state_after = get_git_state() assert is_git_state_same(git_state_before, git_state_after), "Repository state changed after attached HEAD test." return counts @@ -155,9 +155,9 @@ def test_head_detached(patches, counts): git_st_detach_before = git_detach_head() patch_info = patches[0] testid = patch_info["testid"] - results = test(patch_info["root"], patch_info["patch"]) + return_code, results = test(patch_info["root"], patch_info["patch"]) git_st_detach_after = get_git_state() - counts = analyze_result(results, patch_info, counts) + counts = analyze_result(results, patch_info, counts, return_code) if not is_git_state_same(git_st_detach_before, git_st_detach_after): print(" Test '%s' failed with git in detach HEAD mode: state changed after test" % testid.strip(".")) counts["error"] = counts["error"] + 1