From patchwork Wed Mar 18 16:12:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: naftaly.ralamboarivony@smile.fr X-Patchwork-Id: 83759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1266F10775E2 for ; Wed, 18 Mar 2026 16:12:35 +0000 (UTC) Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.18197.1773850350332429256 for ; Wed, 18 Mar 2026 09:12:30 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=bvEXBemN; spf=pass (domain: smile.fr, ip: 209.85.128.51, mailfrom: naftaly.ralamboarivony@smile.fr) Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-4853f2826f7so156995e9.1 for ; Wed, 18 Mar 2026 09:12:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1773850348; x=1774455148; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=59uk4xg/HBz4StSD3zsyEE0VfsVQPxHvu/Tlo8UVCcs=; b=bvEXBemNWpqN/uxGP4c5kT5+T1BAFxDMu5sErsfp3bSNAyaLyxeYqemaQW7SoBNMNF lmbVtLsDdseZz+pLM3hWQhU6cw7TerHLCNzHEVJRF6gmthHSTxxmm5rO3biEaTokZjxO dW7Fkd6/iTpf8zZq5f9g4TKh+xj7frQavkGsA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773850348; x=1774455148; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=59uk4xg/HBz4StSD3zsyEE0VfsVQPxHvu/Tlo8UVCcs=; b=b+iHMAPkVG4VZPpW//BFo+ktareZDQWcqKM4wLw5C1KqQ0jugH44/59OXwU5vopkoc 93F3qRVT/pBfwSXlgBKGQiCuETSdiEsdy+rURESNXS82Qm45huI+Hic3wq52PhYmCS2L i52Wq5A6vfxec1/1JCu9MzOmbuGcRVl5CdRaGBMoDIRYMmBiR6zJCHc2S47svsUdo3H7 Yw/qWARRP8Ecdmwv4gMz02UhRP4IxAMQ7lAtjnY63R2NXSivAaQ0m/L8rvxrJ6tEkj+c KuoXczv+IbSgSIfqoib+iUBS8VWRlN8iCMZaXpNzW+Fpxz6Oe36T95xHzBRB1hYtD5uL 1eLQ== X-Gm-Message-State: AOJu0YwFGnIbWEszTEnSu0EjRObO+Nhwbm0cH7hxHcQDlHDSkhRzFD8v IVHqSx9VgaTNy4YTQi0EiMwM9Tm6XX6C3N4RmnaMFU1Et2vX1f8hGxqvwkZlh9EN/XYxVjWkPIL tH8VW X-Gm-Gg: ATEYQzxyk3BbPusqT59pFGzg083DTiJ643Wn6IinPeHZJMebhMxMuLOucokKFroLuv/ 4hr+k60lXtETBi6bPhlmE0jTixJYpFK/9v42rdVqh2Ah53xr4jnYgdlPookQkpYfFu3aJtbTJMz raTzhPGBkOHUcTajw31+psNZhg5OuCK+l2x4xvdHTGRdesoXzKmFQ0nyxWEh9pcPRp/4GqqPmtP lk9exoBgiGHJvTA8MxBXsSnKNP1XHbWCI2FeKjOq+W0F5Xg5474PsQt8nwU2gzepuBjK1yhFviE CujJnGulaJ5Tg8q4ECXe8t/Vf2h7vIvYyK7DDex4UlOIGh3gmpxBDt2QOr+h+UYF7mqR0f4IctY cUZlH6XsaQekdjxbVeIK6hGyd6UAbSraS+zgn2EnfvsegsKBXxZqikKLLaQ+nO7bUm+jzu71OhJ 72hVD5YBvFtSyYz/s3v0BaVC2DCSWbgp+nRm4/HZJi0ehRA1lUOu70SgT78A5k87BHt4Vr2cuRk 7Kwxpey+W9dQSP/Ji8Ie1cEmI2+2Kz/wejyaC4olKS3Sg== X-Received: by 2002:a05:600c:198f:b0:485:1878:7b8c with SMTP id 5b1f17b1804b1-486f446cfe2mr68902075e9.18.1773850348452; Wed, 18 Mar 2026 09:12:28 -0700 (PDT) Received: from P-ASN-DERVAUX.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-486f45d8e3dsm23629365e9.0.2026.03.18.09.12.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 09:12:28 -0700 (PDT) From: naftaly.ralamboarivony@smile.fr To: openembedded-core@lists.openembedded.org Cc: Naftaly RALAMBOARIVONY Subject: [OE-core][PATCH 1/3] patchtest: add --error-on-failure option Date: Wed, 18 Mar 2026 17:12:07 +0100 Message-Id: <20260318161209.54396-1-naftaly.ralamboarivony@smile.fr> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 16:12:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233412 From: Naftaly RALAMBOARIVONY Add a command line option to make patchtest return a non-zero exit status when a test fails. This keeps the current default behavior unchanged while allowing calling scripts and shells to detect test failures through the process exit status when explicitly requested. Signed-off-by: Naftaly RALAMBOARIVONY --- meta/lib/patchtest/patchtest_parser.py | 3 +++ scripts/patchtest | 8 ++++++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/meta/lib/patchtest/patchtest_parser.py b/meta/lib/patchtest/patchtest_parser.py index 2a11cb76c2..ba5a171e27 100644 --- a/meta/lib/patchtest/patchtest_parser.py +++ b/meta/lib/patchtest/patchtest_parser.py @@ -73,6 +73,9 @@ class PatchtestParser(object): action='store_true', help='Enable logging to a file matching the target patch name with ".testresult" appended') + parser.add_argument('--error-on-failure', + action='store_true', + help='Return non-zero exit status if a test fails') return parser diff --git a/scripts/patchtest b/scripts/patchtest index 9218db232a..653f2b217a 100755 --- a/scripts/patchtest +++ b/scripts/patchtest @@ -183,6 +183,7 @@ def print_result_message(preresult, postresult): print("----------------------------------------------------------------------\n") def main(): + ret = 0 tmp_patch = False patch_path = PatchtestParser.patch_path log_results = PatchtestParser.log_results @@ -214,13 +215,16 @@ def main(): try: if log_path: - run(patch, log_path) + ret = run(patch, log_path) else: - run(patch) + ret = run(patch) finally: if tmp_patch: os.remove(patch) + if PatchtestParser.error_on_failure and ret != 0: + return ret + if __name__ == '__main__': ret = 1 From patchwork Wed Mar 18 16:12:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: naftaly.ralamboarivony@smile.fr X-Patchwork-Id: 83758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2323B10775E1 for ; Wed, 18 Mar 2026 16:12:35 +0000 (UTC) Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.18198.1773850351171554660 for ; Wed, 18 Mar 2026 09:12:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=M3b5gPLU; spf=pass (domain: smile.fr, ip: 209.85.128.41, mailfrom: naftaly.ralamboarivony@smile.fr) Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-485410a0a8aso85285e9.2 for ; Wed, 18 Mar 2026 09:12:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1773850349; x=1774455149; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r1T4hn0lqd8Mr8z7vAGan2FS7DqVqkCkD2sE1KAZk9Q=; b=M3b5gPLUi0Oxvzd0JGVIhv+fQTMzATHIt44zFlBIl2Sv1B61V1Pk69eEcLvQ4Z9W8p cUcAf9m9ZS9rBCU2wKQPGu30ws5K6KQimhOojtFBnks8ljteLyymdnFIWXRYZ6+G6O3A ssc3MC0L+/RncaxFIvGXUqPHKeEybEtbbIolM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773850349; x=1774455149; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=r1T4hn0lqd8Mr8z7vAGan2FS7DqVqkCkD2sE1KAZk9Q=; b=dV/XAwluTNmFxbQqU5d/YUpJ3mKN3PmeXhiA34+TPNehoksNPCgQTqh8K4encteT1r JHNpo3Srfve18Xv2ckS+l07RVFdAYp1YFA64NgWyCtIG8KNXGF8q8NNspkzk9ropuyIw XWqEdf9AyBikrbWYf4ooxLlZPKBlDDPC+RYiyVP1l+2sBODkVHUP9uc8kLXDCCMxBsgX UM9NjSqhCTu3OyxscEncS2+xc44ejPu0IZbKITN8TQnzO1N+SN3CfHTtakKGWLRMlUpp sBu6L7b628QVZS10DZ/i8r3du22b8cCZcYAQgW5J4i5AzGI13kbCUX4aerMabAjvyB/i GcPg== X-Gm-Message-State: AOJu0YxgzKPeGlT+MnsZOcCdYfkgCuN0emEFTT/B+RjFQphwgYd3yYxp 8IZ58c2TNmZGiPmImAxQjYWrBsaIzXawZZsEbuKskWHPfYPTAmAjPAQOi2U/HUjePkakTSo+rkk 6m/Ri X-Gm-Gg: ATEYQzzTFfFQthhQz+pt+JklPLigAuITJ6ugpw355Tie6emvS8Hzua1RVzbt2bkYSSA 7n37FnvuUGXR0lYrRsbs+bU5+BLVRL15agLHCH52HUyZK1ae2wJVYLG/XClB8sZAbJl3ZB3/Gof TEPLDRq5ydZZxDwNxiE/UlUsxW06EhkyCN8mGwwjqSJBVi4m94f0AqZDaOkLuKl5RjBzy1nrf4D ouiu//GSSBrFa3QGp2rVknqtlwxeyCiHFdtPNCplRvbRlIocilBL4BKN26vhRl/84bi2xPUY6ou 2tTRPdQzPLaS6pRP2cxurab8Et79T8Y0LR1meN67QRECRLmfvEv+5mkOgF7lwNw/Q4mCfP4JJ9T 9A2NFJHfNfQRWFkVxyQcyBh0gnqIIUBGazAHhqN0LpqYjCepfTMGxvekVwYSn9IIgX9lEYEnbFy ws1IEBli4KafA/e3kVftmrnbtM1zplcAO2iD7f8cmB94FVzbQAcuQItN+9lWtnrGB/KeS08kmYP ZlJSCz+JDZxAThTTRUm+zUFrEyxUkPNGFXKsIbH1SFgVA== X-Received: by 2002:a05:600c:4fc6:b0:485:38fc:7069 with SMTP id 5b1f17b1804b1-486f4463605mr62610015e9.23.1773850348981; Wed, 18 Mar 2026 09:12:28 -0700 (PDT) Received: from P-ASN-DERVAUX.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-486f45d8e3dsm23629365e9.0.2026.03.18.09.12.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 09:12:28 -0700 (PDT) From: naftaly.ralamboarivony@smile.fr To: openembedded-core@lists.openembedded.org Cc: Naftaly RALAMBOARIVONY Subject: [OE-core][PATCH 2/3] patchtest/selftest: allow passing extra arguments to patchtest Date: Wed, 18 Mar 2026 17:12:08 +0100 Message-Id: <20260318161209.54396-2-naftaly.ralamboarivony@smile.fr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260318161209.54396-1-naftaly.ralamboarivony@smile.fr> References: <20260318161209.54396-1-naftaly.ralamboarivony@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 16:12:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233413 From: Naftaly RALAMBOARIVONY Add an optional extra_args parameter to test() and append it to the patchtest command when provided. Signed-off-by: Naftaly RALAMBOARIVONY --- meta/lib/patchtest/selftest/selftest | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/meta/lib/patchtest/selftest/selftest b/meta/lib/patchtest/selftest/selftest index 43cccf4c85..51681bec38 100755 --- a/meta/lib/patchtest/selftest/selftest +++ b/meta/lib/patchtest/selftest/selftest @@ -130,11 +130,14 @@ def git_detach_head(): return get_git_state() # Once the tests are in oe-core, we can remove the testdir param and use os.path.dirname to get relative paths -def test(root, patch): +def test(root, patch, extra_args=None): res = True patchpath = os.path.abspath(os.path.join(root, patch)) cmd = 'patchtest --base-commit HEAD --repodir %s --testdir %s/tests --patch %s' % (repodir, topdir, patchpath) + if extra_args: + cmd += " " + extra_args + results = subprocess.check_output(cmd, stderr=subprocess.STDOUT, universal_newlines=True, shell=True) return results From patchwork Wed Mar 18 16:12:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: naftaly.ralamboarivony@smile.fr X-Patchwork-Id: 83760 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 441AC10775E6 for ; Wed, 18 Mar 2026 16:12:35 +0000 (UTC) Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.18199.1773850351581220196 for ; Wed, 18 Mar 2026 09:12:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=rFgsFLUy; spf=pass (domain: smile.fr, ip: 209.85.128.49, mailfrom: naftaly.ralamboarivony@smile.fr) Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-48541edecf9so219995e9.1 for ; Wed, 18 Mar 2026 09:12:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1773850350; x=1774455150; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=R4o7c9o2FdC3GK7EjnihSo+RuuiOVSZkdQNMJSLNo2M=; b=rFgsFLUyZNlbRDuAa1JXulawRvMglRDa1eXpV4KE19hRHcv3E/kt+HWLVamaZrH2K0 Z5KlXuR7y9+cPSy8AgL+JiOoeOgjbVjyOZcYvf1l+NS2NLlnXLg8bAD1iy/bETBGir5K nNOszgwx+INeYcmp3G544mBF0+5y98bv6OrsQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773850350; x=1774455150; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=R4o7c9o2FdC3GK7EjnihSo+RuuiOVSZkdQNMJSLNo2M=; b=tVotUiL6gE5wn6Xw0Bsbh+7JTPCsgct8VTDqyakC773fMBjGl/l7zgEUfZmR/WJBZP PSJnrrdMwFfN7CJxbzrka+1p1nMwZi6TsiMtts2gH0RVdZ52p8/NmOSvLlCk6PMEXcUi Z3iaGJHZkNv0SFy1vRdI/oI1nkEfpRD1gCzpAEdl8nhKE+1LduC/GPjl2I+oDuYn9QRv sNtivOAeqIeLZuTCq99GUAvH8+sfJ4yRN2lBV2Tq8Wb21QefakIGkie+mHBrBcbk7Jdo kvEEKBymzZJbaAVhS1XCKN0SQ4V5i7ESxLYlg85wTL9hTjrYxvoFjlcTOePLxnyZUb8K xP9A== X-Gm-Message-State: AOJu0YzCeSCQXYHFjSfUYL4RSz5k9uRsaGznFQ6N6wMtkA9D/M02PSgI bu3IP3cZ+GIbxfsRQNCGnZZ8zI8uFBn3cgK5SAq8gJQcYYXYK4HJJYRvYrAf8afOvhU1YVGen3W c1s5j X-Gm-Gg: ATEYQzwGFNVjT/EVIqpYL7WR4mopoQw8QoLJLadhh7coKKyITPqYu+wza4pNvNQqnOZ lDQCNiWXSOf0Pz1otvR4pMmF2JiBuNy7rZ9V164Lx32dED7TomZFO63cPCWDxEPslaW3HXMUzPB FX0l/m0dvmR2kIzAxnCWwhbIdhLNo9l+2FxI6U51ZEz8rLzMngaD6LSjvxYviZncfyLS1XTb4xE +fBL+FlQAERpGH3WVXBgE5zQBnO4nYZOrppexU1zpxrx5EkNmoekmtvvc0tW/SZdA2np7CTJvyu +evqqsb3axrzwpRrPktr0o0ctuyjOnCITM046q85lYWGQxP7AMTDk312oAKtj2CSpEvB1gJbzUm 8gcR87I/BOU65z8cDeVKIgBGmKtF1jMm7oJI9yMOpLb/zpj5nug0qBGmV+NBYEXxKXc3ux8//VG RTdUvQmnXSTvPPdE0+jEzoClQ5+l7gS8G6SbmZF5SVWqzNECLYW7vjfshvlfs7O+3ZLJBTKajRH bngZsLSL0iYXC/nOIC+SaqLdPOkheNi3Z1aN9rdRLUIAA== X-Received: by 2002:a05:600d:8495:10b0:485:40c6:f51a with SMTP id 5b1f17b1804b1-486f4465dfbmr51147485e9.31.1773850349767; Wed, 18 Mar 2026 09:12:29 -0700 (PDT) Received: from P-ASN-DERVAUX.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-486f45d8e3dsm23629365e9.0.2026.03.18.09.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 09:12:29 -0700 (PDT) From: naftaly.ralamboarivony@smile.fr To: openembedded-core@lists.openembedded.org Cc: Naftaly RALAMBOARIVONY Subject: [OE-core][PATCH 3/3] patchtest/selftest: add test case for --error-on-failure return code Date: Wed, 18 Mar 2026 17:12:09 +0100 Message-Id: <20260318161209.54396-3-naftaly.ralamboarivony@smile.fr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260318161209.54396-1-naftaly.ralamboarivony@smile.fr> References: <20260318161209.54396-1-naftaly.ralamboarivony@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 16:12:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233414 From: Naftaly RALAMBOARIVONY Add a test case to verify that patchtest --error-on-failure returns the correct exit code for PASS,FAIL and SKIP scenarios. Signed-off-by: Naftaly RALAMBOARIVONY --- meta/lib/patchtest/selftest/selftest | 37 ++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/meta/lib/patchtest/selftest/selftest b/meta/lib/patchtest/selftest/selftest index 51681bec38..39114a45a0 100755 --- a/meta/lib/patchtest/selftest/selftest +++ b/meta/lib/patchtest/selftest/selftest @@ -142,6 +142,42 @@ def test(root, patch, extra_args=None): return results +def test_error_on_failure(root, patch): + try: + output = test(root, patch, "--error-on-failure") + return output, 0 + except subprocess.CalledProcessError as e: + return e.output, e.returncode + +def test_returncode(patches, counts): + + for target in ("FAIL", "PASS", "SKIP"): + for patch_info in patches: + + testid = patch_info["testid"] + expected = str(patch_info["expected"]).upper() + + if expected == target: + results, returncode = test_error_on_failure( + patch_info["root"], patch_info["patch"] + ) + + if target == "FAIL" and returncode != 0: + print("XFAIL: test_returncode.%s (file: %s)" % (testid.strip("."), os.path.basename(patch_info["patch"]))) + counts["xfail"] = counts["xfail"] + 1 + elif target == "PASS" and returncode == 0: + counts["xpass"] = counts["xpass"] + 1 + print("XPASS: test_returncode.%s (file: %s)" % (testid.strip("."), os.path.basename(patch_info["patch"]))) + elif target == "SKIP" and returncode == 0: + counts["xskip"] = counts["xskip"] + 1 + print("XSKIP: test_returncode.%s (file: %s)" % (testid.strip("."), os.path.basename(patch_info["patch"]))) + else: + print(f"Test failed: target '{target}', expected return code '{expected}', got '{returncode}'") + + break + + return counts + def test_head_attached(patches, counts, branch): git_attach_head(branch) @@ -178,6 +214,7 @@ def run_tests(patches, counts): counts = test_head_detached(patches, counts) restore_git_state(git_state) run_sh(f"git branch -D {temp_branch}") + counts = test_returncode(patches, counts) return counts