From patchwork Fri Jun 5 04:58:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anders Heimer X-Patchwork-Id: 89335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52D8BCD6E79 for ; Fri, 5 Jun 2026 04:58:39 +0000 (UTC) Received: from DUZPR83CU001.outbound.protection.outlook.com (DUZPR83CU001.outbound.protection.outlook.com [52.101.66.29]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.1519.1780635511989931224 for ; Thu, 04 Jun 2026 21:58:33 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@est.tech header.s=selector1 header.b=vla7C12T; spf=pass (domain: est.tech, ip: 52.101.66.29, mailfrom: anders.heimer@est.tech) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=E1d1Ye00zafQENalYe6TzJJjlaZDKbpt+wmEZa9BTc5J7xcAUKj3YkwAkyV/WuDc9XuIy6v+eSxKVab0lAE5ZBc5VvZZFZEyW91EllBGs4WWMuxztn+KvFMJqwkjV0v1+aY2+MDmK34jRM1YIvb0wmCeERkYyOXcguGfMZM/RLgofTiMyDlygN+aaW7fW9/bj+5bUQJ6rVpWkhy3qNrudXoJweP0GC2V9r43PSQPe/2kxpcYShU9Qh/NeYtSBHJYDQ2a/lLct+Vrfb8lRdoSTIBXZpHu8Iw526NS8esCc/dt0podpMpoE60bcMLLqbgX6ERPkvQF3NvSYZs+OQ/cyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AcTlonNUyQeLyskhT67sW1FTcHMexp+HnXRHYqJ2liw=; b=vsxfr8LvVp88Lq5B8IqpJ1LRvQ7bWdXqJ9qHqCxjDpHHfnIyygmuCKHLCxaRrMFpwDHpOq6MBW/ciEt8fu6xUbtWIcsE69Z9ohHWsNqXOewgoWLIMGnmSpMpUfJBEugyFbuQowEd9kycKOUe7F8kHUsysFLqxX5JmY6hpXWeR+QZuO+2K1wW5KRAwCCVB8cc08RVF7xUziBzAChr/1rm0xMrQRKJvo86moM4RcLJ7T1LVDKYeMSrEpgWq88EqwNumUls0OInV2PkEVgzlqeGMrVF1FqJoX6myogAfMI6rJAYP1psxlGPoBRX/dz/6kiFHjCjDK/QE/a9fAIZqYeyBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=est.tech; dmarc=pass action=none header.from=est.tech; dkim=pass header.d=est.tech; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=est.tech; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AcTlonNUyQeLyskhT67sW1FTcHMexp+HnXRHYqJ2liw=; b=vla7C12TaROsk30Rxri4I2Ed5F7BXX5s8tiWOUhl5MNhryTClwoOKpjO5kkLJ7sz1x2VRuWagDl1P03xhUiztM3QtOyscsTeHwAnIl6Vz0jAxRR622y0RLwP+r//P3XumCTEmoBO08IIyewqAL9QU0L7ML7xIhCphVt6y0y14bgfCD6H8RCHluFvgkzLPPE1SG6S+dcgpLBb1rlV2B7vioDJuH7JlEdEwpQzYVgyAKATErG9AvaF/GlFxhEjHL/7uDV0wbp+2LkAVZoNyw12w0QjlxDfnKTpz40zz1xvxWajGV0JQqe1Lvphc2Utm1ke2YPJKnCeubZRhR9azmr2BA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=est.tech; Received: from DB9P189MB1641.EURP189.PROD.OUTLOOK.COM (2603:10a6:10:2ac::9) by AM0P189MB0785.EURP189.PROD.OUTLOOK.COM (2603:10a6:208:19f::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.8; Fri, 5 Jun 2026 04:58:29 +0000 Received: from DB9P189MB1641.EURP189.PROD.OUTLOOK.COM ([fe80::90da:b700:f102:5c82]) by DB9P189MB1641.EURP189.PROD.OUTLOOK.COM ([fe80::90da:b700:f102:5c82%6]) with mapi id 15.21.0092.007; Fri, 5 Jun 2026 04:58:29 +0000 From: Anders Heimer To: bitbake-devel@lists.openembedded.org Subject: [PATCH 1/2] hashserv/tests: use valid 64-character unihashes Date: Fri, 5 Jun 2026 06:58:25 +0200 Message-ID: <20260605045826.2052909-2-anders.heimer@est.tech> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260605045826.2052909-1-anders.heimer@est.tech> References: <20260605045826.2052909-1-anders.heimer@est.tech> X-ClientProxiedBy: DU7PR01CA0042.eurprd01.prod.exchangelabs.com (2603:10a6:10:50e::26) To DB9P189MB1641.EURP189.PROD.OUTLOOK.COM (2603:10a6:10:2ac::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB9P189MB1641:EE_|AM0P189MB0785:EE_ X-MS-Office365-Filtering-Correlation-Id: 695ec6ed-c2ad-4247-4e56-08dec2bf157a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016|22082099003|18002099003|3023799007|56012099006|11063799006; X-Microsoft-Antispam-Message-Info: jLN1YP5+1zL5ySFWpgmUJdPfXsfqZB0Npkcj0gjZrCnMmmQ3u64lOCnl4pTpfhQzNb3hAbAnXTa/9Y8mC/j7IgSoTFPKTedfQPv+D/xepr3MMCYJtr/bpNTg0OtGIXu+oFt8otggSBH3vqvsBrT2/QFHoFQHSj4kavsHaQmdFsTwocTOEOJVtoNPIwFHX1wik8TV2X2T2McC6jkYgYF77VLMhPGM5+hkAYYBQey2CHQZo4aYYE2UraZVwBkmr0ac6OcO/fBRgLVVZwUOvESL+0fGNDHe26aE+/JiJCc1pq9rQDywysADrDl1s/VrrHOIaxpE8e2NsUllCjkr4sZ5ruX1qTgFdx1pnAf4HHOJmMkOwFk+yRPTAcNSlytzwZTPylCIu4lvLhJ6zQXpDxe6lUTVN6G4g2DPPU22t6KR9vqNbej74qWoeyV5PLtIe8igvbSwrvNuI+/oyw6M5vByYn3ADSW7ZjTkT+pN/U1CMAKnqn8Obn9qiCrke4RbiCmV//d5F4MMldD1TQjSgjchO26BdbMnKoX/+5liOipNh/8hV3Z7d6k0SdWo68a5fs/k2AEMnsEmeze5xGdqGjqGxmqH9ASFKXi/qmkqYnrBPreVUkCKIEvpUXLnNNWRqlk5N8TrElOYFqFcVIlxb6QTHO5ZzxwnQE27JfgK+zxUKG+dP+CLrxPtbnZkx1ZyzJxD X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9P189MB1641.EURP189.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016)(22082099003)(18002099003)(3023799007)(56012099006)(11063799006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: I84tJDX5oe8iYlwgguuDfa6NnXMS9O+DxgqQBfIM8yHrnwDS5FLYs/kx91wfyAYV5GMlCwx1aqGmp7tJliEXLUZ+8C1UpnJ+Txcwp3p6GWKUmouQLC0X423p17Thl2RqM3ykFAgBHozR1zvnecrbjjM1Ph+tyRos255WJcykXu7ouO/7PzlGibU48GWTzewsH9Az6Xfaypgu1+tRqQNz9J8DyVKBaJJyTd2TaFtNk34GBQQb+ZtZMOY2QueRfzcA/GWLwbLcDoQdAGg+bKPg0gjq6d3ARns0Al8eh9V/XVn3AwkG1RMW6Rghtq80zmvoWCaAzsTLbkMTl05Gj9Cpwuz9eSkdfiWSkCb6W+03trnHU5xkrDDscZJPfIqSEPO2zux77Q0tBS0KziWpfea/fpUHvMYeJHg3PDXD2DRBWvm8aW8W3CzdY6SEgPu08UzTJmvqd/FGzzsRN5Ed1+jngHTx9sdWLpiH2ulNXRnsosd4b+0BrgMU7Z5nmHVq/VPQQp+S5hSBW1cZc9kf72eLhL0DpJbDABBlJY5qEIp76yue/YqncdtaPZ6j8Pf1AEtsd+xRkM/mxkZXRc4Dp2r+pQOINrmjWCBC3ZoZBmg3QoAs/5XlwamPAIqzk36yWUnamvCXGm1ZttTTUcOZFqCOUGDpoTclIhtcKUfU4xC7AFqiJblkyw3TGDkAU/sRAxLgVLFI9lH0E8iWW4EHpxsgidIBXBY6xnXrQsJyEKQG5ape5yZA7M+NNC7sAH8T9T3GjLhHIoIXMFZ5YChpL7BWPD9UYNw9XjDcHQIWI2K2n6LOZXZYyyJhAXIGJsiyqqQ1JGquojwe6sg+WrPkx81ZZS6c8stiN/E9+Lt97z2lkGiYualdDUhKnHOenL0JzAPQWHKU9ER6fQqbfiO1zTSGy8Kuqe/wio1le6aLq71bXLgijMEVlphAzd0j0BVRdchh3buOsuZbaWUzmNT+fBDdTaiSX9V+hN6aFiq08pPHujZ4LsjuUA2Y23Bx7z4q69rulvz5Y8jFV1RoonwmTaBSacMhJ/6v9a6WIhHs+LTY5gMgy2ZrglynTktS/4EFtm273tt4sk9goZsDo0QMZZ5KHXm3TypUAKXiUivbLpyB/Kb7eRYrvD8MahyjPcTb0r1eB0K5FWoIT5mVGqzj1vp9vuNdysF54+YOMBJvGZ0nrT9qZvEKU/HmGRP8EaDYGypJ42DU9YjVZzLhxCvRlle2gUD3H/2S9kUhU7CaRjdR+nYpKVhj5G1zvVHX6w6Y/n6Cf1CxJlGCD1buejP1BqLbbN0sH/EMI132HjG5HlIIXuoegPhr21r/EDrM+WLHwi6nE6QG9tBrmetw0sbD+kGyqqonfNiqaui3piPYMBJ7pL+MPYLcJdgCGLZNQmwvjYDZWyu6EmsL0DaZuDFjvofubGprAFqFviW7rRiICUir1JsJMNpMsW3j2cGqzH3/TlSsLYKRMKb+ikCtIKrge1EB0+KaUk+RAo4LQ4H2ObzqngFrRZo6/8kew241FG3blidtGXCgMrHYx+3vvTSdUkUA7uwclBVfyjiOXV15SkzukmyIfKvDEBsK2Q+ctsi1Ipj8eZXWaE2chfS7TOJqNjb0vvpuQo98JIzu7gKAMkKKLO6escFAX6MO7d0ClE6amR5+8Dpf0Ud4pNSZXInjaYng1cwLV6Bf/EI/xhIbTukm0SZAFTzi+tSyxsk+llML7ifGgbvJN+tSDOjgtKtCNZwEUA== X-OriginatorOrg: est.tech X-MS-Exchange-CrossTenant-Network-Message-Id: 695ec6ed-c2ad-4247-4e56-08dec2bf157a X-MS-Exchange-CrossTenant-AuthSource: DB9P189MB1641.EURP189.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2026 04:58:29.2830 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: d2585e63-66b9-44b6-a76e-4f4b217d97fd X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VFzVsYPBfNjqB8pTUZJaLHYl5kOKjsVjPpRXKsUTI2RMFaTFhe2T1j5aF1CU9Lko3d/qybsi2/PZXREeio8Axw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0P189MB0785 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 05 Jun 2026 04:58:39 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19613 Signed-off-by: Anders Heimer --- lib/hashserv/tests.py | 164 ++++++++++++++++++++++++++++-------------- 1 file changed, 110 insertions(+), 54 deletions(-) diff --git a/lib/hashserv/tests.py b/lib/hashserv/tests.py index 124d8aa00..83ce0c5ca 100644 --- a/lib/hashserv/tests.py +++ b/lib/hashserv/tests.py @@ -28,6 +28,7 @@ from pathlib import Path THIS_DIR = Path(__file__).parent BIN_DIR = THIS_DIR.parent.parent / "bin" + def server_prefunc(server, idx): logging.basicConfig(level=logging.DEBUG, filename='bbhashserv-%d.log' % idx, filemode='w', format='%(levelname)s %(filename)s:%(lineno)d %(message)s') @@ -124,7 +125,7 @@ class HashEquivalenceTestSetup(object): # Simple test that hashes can be created taskhash = '35788efcb8dfb0a02659d81cf2bfd695fb30faf9' outhash = '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f' - unihash = 'f46d3fbb439bd9b921095da657a4de906510d2cd' + unihash = 'a69ec97f5af2e21e1a1f9cc8896965515d5559425666f734e245a3d40cee33d9' self.assertClientGetHash(client, taskhash, None) @@ -178,7 +179,7 @@ class HashEquivalenceCommonTests(object): # assigned the same unihash taskhash = '53b8dce672cb6d0c73170be43f540460bfc347b4' outhash = '5a9cb1649625f0bf41fc7791b635cd9c2d7118c7f021ba87dcd03f72b67ce7a8' - unihash = 'f37918cc02eb5a520b1aff86faacbc0a38124646' + unihash = '46edb5140d2613049332d0bf3745d9fafec9c559dac8cc61813739a28007fcdf' result = self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') @@ -186,7 +187,7 @@ class HashEquivalenceCommonTests(object): # Report a different task with the same outhash. The returned unihash # should match the first task taskhash2 = '3bf6f1e89d26205aec90da04854fbdbf73afe6b4' - unihash2 = 'af36b199320e611fbb16f1f277d3ee1d619ca58b' + unihash2 = 'bf6e81926066f770e960f9f777cd088c62bea9addb7745f3e77deaa81a645747' result = self.client.report_unihash(taskhash2, self.METHOD, outhash, unihash2) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') @@ -196,19 +197,19 @@ class HashEquivalenceCommonTests(object): # taskhash taskhash = '8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a' outhash = 'afe240a439959ce86f5e322f8c208e1fedefea9e813f2140c81af866cc9edf7e' - unihash = '218e57509998197d570e2c98512d0105985dffc9' + unihash = '5b521d8a12683086cc08bc2c6d94a7a2dcff17eba53b9911e145d51164689380' self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertClientGetHash(self.client, taskhash, unihash) outhash2 = '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d' - unihash2 = 'ae9a7d252735f0dafcdb10e2e02561ca3a47314c' + unihash2 = 'a37541b54fd22440e292f617eb30ba07455e88fb0b9f0952eca229b6356290e3' self.client.report_unihash(taskhash, self.METHOD, outhash2, unihash2) self.assertClientGetHash(self.client, taskhash, unihash) outhash3 = '77623a549b5b1a31e3732dfa8fe61d7ce5d44b3370f253c5360e136b852967b4' - unihash3 = '9217a7d6398518e5dc002ed58f2cbbbc78696603' + unihash3 = '6842f1f2daccd96ddef15c9154d4e41ac8a2300d781ac9a9db7f8afeb8a96808' self.client.report_unihash(taskhash, self.METHOD, outhash3, unihash3) self.assertClientGetHash(self.client, taskhash, unihash) @@ -268,7 +269,7 @@ class HashEquivalenceCommonTests(object): # Simple test that hashes can be created taskhash = 'c665584ee6817aa99edfc77a44dd853828279370' outhash = '3c979c3db45c569f51ab7626a4651074be3a9d11a84b1db076f5b14f7d39db44' - unihash = '90e9bc1d1f094c51824adca7f8ea79a048d68824' + unihash = '06f89b8f329ba8124ff73c56d09ef921b42624747c421277bddaf5e23f136e57' self.assertClientGetHash(self.client, taskhash, None) @@ -361,7 +362,7 @@ class HashEquivalenceCommonTests(object): # Basic report taskhash = '8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a' outhash = 'afe240a439959ce86f5e322f8c208e1fedefea9e813f2140c81af866cc9edf7e' - unihash = '218e57509998197d570e2c98512d0105985dffc9' + unihash = '5b521d8a12683086cc08bc2c6d94a7a2dcff17eba53b9911e145d51164689380' self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) check_hash(taskhash, unihash, None) @@ -369,7 +370,7 @@ class HashEquivalenceCommonTests(object): # Duplicated taskhash with multiple output hashes and unihashes. # All servers should agree with the originally reported hash outhash2 = '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d' - unihash2 = 'ae9a7d252735f0dafcdb10e2e02561ca3a47314c' + unihash2 = 'a37541b54fd22440e292f617eb30ba07455e88fb0b9f0952eca229b6356290e3' self.client.report_unihash(taskhash, self.METHOD, outhash2, unihash2) check_hash(taskhash, unihash, unihash) @@ -377,7 +378,7 @@ class HashEquivalenceCommonTests(object): # Report an equivalent task. The sideload will originally report # no unihash until backfilled taskhash3 = "044c2ec8aaf480685a00ff6ff49e6162e6ad34e1" - unihash3 = "def64766090d28f627e816454ed46894bb3aab36" + unihash3 = "aca636d800aef40e6ddcea4b2262cc4ea0d1180a6783e5b4653a20c7dd73458d" self.client.report_unihash(taskhash3, self.METHOD, outhash, unihash3) check_hash(taskhash3, unihash, None) @@ -386,7 +387,7 @@ class HashEquivalenceCommonTests(object): # propagating to the upstream server taskhash4 = "e3da00593d6a7fb435c7e2114976c59c5fd6d561" outhash4 = "1cf8713e645f491eb9c959d20b5cae1c47133a292626dda9b10709857cbe688a" - unihash4 = "3b5d3d83f07f259e9086fcb422c855286e18a57d" + unihash4 = "7aebef07d66a8c0f92d0c4f65ec8b1fbb850a3693c53827b8774b64fa9a8a9fe" down_client.report_unihash(taskhash4, self.METHOD, outhash4, unihash4) down_client.backfill_wait() @@ -398,18 +399,18 @@ class HashEquivalenceCommonTests(object): # match which was previously reported to the upstream server taskhash5 = '35788efcb8dfb0a02659d81cf2bfd695fb30faf9' outhash5 = '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f' - unihash5 = 'f46d3fbb439bd9b921095da657a4de906510d2cd' + unihash5 = 'a69ec97f5af2e21e1a1f9cc8896965515d5559425666f734e245a3d40cee33d9' result = self.client.report_unihash(taskhash5, self.METHOD, outhash5, unihash5) taskhash6 = '35788efcb8dfb0a02659d81cf2bfd695fb30fafa' - unihash6 = 'f46d3fbb439bd9b921095da657a4de906510d2ce' + unihash6 = 'eabc7a98e0c12bbeb8394dbdf055eb81aac60e4a14cca5c1f069d36efc933b23' result = down_client.report_unihash(taskhash6, self.METHOD, outhash5, unihash6) self.assertEqual(result['unihash'], unihash5, 'Server failed to copy unihash from upstream') # Tests read through from server with taskhash7 = '9d81d76242cc7cfaf7bf74b94b9cd2e29324ed74' outhash7 = '8470d56547eea6236d7c81a644ce74670ca0bbda998e13c629ef6bb3f0d60b69' - unihash7 = '05d2a63c81e32f0a36542ca677e8ad852365c538' + unihash7 = '7521a98a0c645341bc51559b234ef37a097e8f3a01665e0303a317925ab7b4d5' self.client.report_unihash(taskhash7, self.METHOD, outhash7, unihash7) result = down_client.get_taskhash(self.METHOD, taskhash7, True) @@ -420,7 +421,7 @@ class HashEquivalenceCommonTests(object): taskhash8 = '86978a4c8c71b9b487330b0152aade10c1ee58aa' outhash8 = 'ca8c128e9d9e4a28ef24d0508aa20b5cf880604eacd8f65c0e366f7e0cc5fbcf' - unihash8 = 'd8bcf25369d40590ad7d08c84d538982f2023e01' + unihash8 = '83386d9385b0bf3ba25693127ddcaaadeaa1c4bf8cb0baecfb5314b9a20072a1' self.client.report_unihash(taskhash8, self.METHOD, outhash8, unihash8) result = down_client.get_outhash(self.METHOD, outhash8, taskhash8) @@ -431,7 +432,7 @@ class HashEquivalenceCommonTests(object): taskhash9 = 'ae6339531895ddf5b67e663e6a374ad8ec71d81c' outhash9 = 'afc78172c81880ae10a1fec994b5b4ee33d196a001a1b66212a15ebe573e00b5' - unihash9 = '6662e699d6e3d894b24408ff9a4031ef9b038ee8' + unihash9 = 'cc74784b2c0ad5b378a6b783c74c518d2c46b8b52fba29cb39a8430d742440d7' self.client.report_unihash(taskhash9, self.METHOD, outhash9, unihash9) result = down_client.get_taskhash(self.METHOD, taskhash9, False) @@ -454,7 +455,7 @@ class HashEquivalenceCommonTests(object): # Report a hash via the read-write server taskhash = '35788efcb8dfb0a02659d81cf2bfd695fb30faf9' outhash = '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f' - unihash = 'f46d3fbb439bd9b921095da657a4de906510d2cd' + unihash = 'a69ec97f5af2e21e1a1f9cc8896965515d5559425666f734e245a3d40cee33d9' result = rw_client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') @@ -465,7 +466,7 @@ class HashEquivalenceCommonTests(object): # Ensure that reporting via the read-only server fails taskhash2 = 'c665584ee6817aa99edfc77a44dd853828279370' outhash2 = '3c979c3db45c569f51ab7626a4651074be3a9d11a84b1db076f5b14f7d39db44' - unihash2 = '90e9bc1d1f094c51824adca7f8ea79a048d68824' + unihash2 = '06f89b8f329ba8124ff73c56d09ef921b42624747c421277bddaf5e23f136e57' result = ro_client.report_unihash(taskhash2, self.METHOD, outhash2, unihash2) self.assertEqual(result['unihash'], unihash2) @@ -554,15 +555,43 @@ class HashEquivalenceCommonTests(object): def test_get_unihash_batch(self): TEST_INPUT = ( # taskhash outhash unihash - ('8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a', 'afe240a439959ce86f5e322f8c208e1fedefea9e813f2140c81af866cc9edf7e','218e57509998197d570e2c98512d0105985dffc9'), + ( + '8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a', + 'afe240a439959ce86f5e322f8c208e1fedefea9e813f2140c81af866cc9edf7e', + '5b521d8a12683086cc08bc2c6d94a7a2dcff17eba53b9911e145d51164689380', + ), # Duplicated taskhash with multiple output hashes and unihashes. - ('8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a', '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d', 'ae9a7d252735f0dafcdb10e2e02561ca3a47314c'), + ( + '8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a', + '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d', + 'a37541b54fd22440e292f617eb30ba07455e88fb0b9f0952eca229b6356290e3', + ), # Equivalent hash - ("044c2ec8aaf480685a00ff6ff49e6162e6ad34e1", '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d', "def64766090d28f627e816454ed46894bb3aab36"), - ("e3da00593d6a7fb435c7e2114976c59c5fd6d561", "1cf8713e645f491eb9c959d20b5cae1c47133a292626dda9b10709857cbe688a", "3b5d3d83f07f259e9086fcb422c855286e18a57d"), - ('35788efcb8dfb0a02659d81cf2bfd695fb30faf9', '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f', 'f46d3fbb439bd9b921095da657a4de906510d2cd'), - ('35788efcb8dfb0a02659d81cf2bfd695fb30fafa', '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f', 'f46d3fbb439bd9b921095da657a4de906510d2ce'), - ('9d81d76242cc7cfaf7bf74b94b9cd2e29324ed74', '8470d56547eea6236d7c81a644ce74670ca0bbda998e13c629ef6bb3f0d60b69', '05d2a63c81e32f0a36542ca677e8ad852365c538'), + ( + "044c2ec8aaf480685a00ff6ff49e6162e6ad34e1", + '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d', + "aca636d800aef40e6ddcea4b2262cc4ea0d1180a6783e5b4653a20c7dd73458d", + ), + ( + "e3da00593d6a7fb435c7e2114976c59c5fd6d561", + "1cf8713e645f491eb9c959d20b5cae1c47133a292626dda9b10709857cbe688a", + "7aebef07d66a8c0f92d0c4f65ec8b1fbb850a3693c53827b8774b64fa9a8a9fe", + ), + ( + '35788efcb8dfb0a02659d81cf2bfd695fb30faf9', + '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f', + 'a69ec97f5af2e21e1a1f9cc8896965515d5559425666f734e245a3d40cee33d9', + ), + ( + '35788efcb8dfb0a02659d81cf2bfd695fb30fafa', + '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f', + 'eabc7a98e0c12bbeb8394dbdf055eb81aac60e4a14cca5c1f069d36efc933b23', + ), + ( + '9d81d76242cc7cfaf7bf74b94b9cd2e29324ed74', + '8470d56547eea6236d7c81a644ce74670ca0bbda998e13c629ef6bb3f0d60b69', + '7521a98a0c645341bc51559b234ef37a097e8f3a01665e0303a317925ab7b4d5', + ), ) EXTRA_QUERIES = ( "6b6be7a84ab179b4240c4302518dc3f6", @@ -578,28 +607,56 @@ class HashEquivalenceCommonTests(object): ) self.assertListEqual(result, [ - "218e57509998197d570e2c98512d0105985dffc9", - "218e57509998197d570e2c98512d0105985dffc9", - "218e57509998197d570e2c98512d0105985dffc9", - "3b5d3d83f07f259e9086fcb422c855286e18a57d", - "f46d3fbb439bd9b921095da657a4de906510d2cd", - "f46d3fbb439bd9b921095da657a4de906510d2cd", - "05d2a63c81e32f0a36542ca677e8ad852365c538", + "5b521d8a12683086cc08bc2c6d94a7a2dcff17eba53b9911e145d51164689380", + "5b521d8a12683086cc08bc2c6d94a7a2dcff17eba53b9911e145d51164689380", + "5b521d8a12683086cc08bc2c6d94a7a2dcff17eba53b9911e145d51164689380", + "7aebef07d66a8c0f92d0c4f65ec8b1fbb850a3693c53827b8774b64fa9a8a9fe", + "a69ec97f5af2e21e1a1f9cc8896965515d5559425666f734e245a3d40cee33d9", + "a69ec97f5af2e21e1a1f9cc8896965515d5559425666f734e245a3d40cee33d9", + "7521a98a0c645341bc51559b234ef37a097e8f3a01665e0303a317925ab7b4d5", None, ]) def test_unihash_exists_batch(self): TEST_INPUT = ( # taskhash outhash unihash - ('8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a', 'afe240a439959ce86f5e322f8c208e1fedefea9e813f2140c81af866cc9edf7e','218e57509998197d570e2c98512d0105985dffc9'), + ( + '8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a', + 'afe240a439959ce86f5e322f8c208e1fedefea9e813f2140c81af866cc9edf7e', + '5b521d8a12683086cc08bc2c6d94a7a2dcff17eba53b9911e145d51164689380', + ), # Duplicated taskhash with multiple output hashes and unihashes. - ('8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a', '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d', 'ae9a7d252735f0dafcdb10e2e02561ca3a47314c'), + ( + '8aa96fcffb5831b3c2c0cb75f0431e3f8b20554a', + '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d', + 'a37541b54fd22440e292f617eb30ba07455e88fb0b9f0952eca229b6356290e3', + ), # Equivalent hash - ("044c2ec8aaf480685a00ff6ff49e6162e6ad34e1", '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d', "def64766090d28f627e816454ed46894bb3aab36"), - ("e3da00593d6a7fb435c7e2114976c59c5fd6d561", "1cf8713e645f491eb9c959d20b5cae1c47133a292626dda9b10709857cbe688a", "3b5d3d83f07f259e9086fcb422c855286e18a57d"), - ('35788efcb8dfb0a02659d81cf2bfd695fb30faf9', '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f', 'f46d3fbb439bd9b921095da657a4de906510d2cd'), - ('35788efcb8dfb0a02659d81cf2bfd695fb30fafa', '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f', 'f46d3fbb439bd9b921095da657a4de906510d2ce'), - ('9d81d76242cc7cfaf7bf74b94b9cd2e29324ed74', '8470d56547eea6236d7c81a644ce74670ca0bbda998e13c629ef6bb3f0d60b69', '05d2a63c81e32f0a36542ca677e8ad852365c538'), + ( + "044c2ec8aaf480685a00ff6ff49e6162e6ad34e1", + '0904a7fe3dc712d9fd8a74a616ddca2a825a8ee97adf0bd3fc86082c7639914d', + "aca636d800aef40e6ddcea4b2262cc4ea0d1180a6783e5b4653a20c7dd73458d", + ), + ( + "e3da00593d6a7fb435c7e2114976c59c5fd6d561", + "1cf8713e645f491eb9c959d20b5cae1c47133a292626dda9b10709857cbe688a", + "7aebef07d66a8c0f92d0c4f65ec8b1fbb850a3693c53827b8774b64fa9a8a9fe", + ), + ( + '35788efcb8dfb0a02659d81cf2bfd695fb30faf9', + '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f', + 'a69ec97f5af2e21e1a1f9cc8896965515d5559425666f734e245a3d40cee33d9', + ), + ( + '35788efcb8dfb0a02659d81cf2bfd695fb30fafa', + '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f', + 'eabc7a98e0c12bbeb8394dbdf055eb81aac60e4a14cca5c1f069d36efc933b23', + ), + ( + '9d81d76242cc7cfaf7bf74b94b9cd2e29324ed74', + '8470d56547eea6236d7c81a644ce74670ca0bbda998e13c629ef6bb3f0d60b69', + '7521a98a0c645341bc51559b234ef37a097e8f3a01665e0303a317925ab7b4d5', + ), ) EXTRA_QUERIES = ( "6b6be7a84ab179b4240c4302518dc3f6", @@ -939,14 +996,14 @@ class HashEquivalenceCommonTests(object): def test_gc(self): taskhash = '53b8dce672cb6d0c73170be43f540460bfc347b4' outhash = '5a9cb1649625f0bf41fc7791b635cd9c2d7118c7f021ba87dcd03f72b67ce7a8' - unihash = 'f37918cc02eb5a520b1aff86faacbc0a38124646' + unihash = '46edb5140d2613049332d0bf3745d9fafec9c559dac8cc61813739a28007fcdf' result = self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') taskhash2 = '3bf6f1e89d26205aec90da04854fbdbf73afe6b4' outhash2 = '77623a549b5b1a31e3732dfa8fe61d7ce5d44b3370f253c5360e136b852967b4' - unihash2 = 'af36b199320e611fbb16f1f277d3ee1d619ca58b' + unihash2 = 'bf6e81926066f770e960f9f777cd088c62bea9addb7745f3e77deaa81a645747' result = self.client.report_unihash(taskhash2, self.METHOD, outhash2, unihash2) self.assertClientGetHash(self.client, taskhash2, unihash2) @@ -972,21 +1029,21 @@ class HashEquivalenceCommonTests(object): def test_gc_stream(self): taskhash = '53b8dce672cb6d0c73170be43f540460bfc347b4' outhash = '5a9cb1649625f0bf41fc7791b635cd9c2d7118c7f021ba87dcd03f72b67ce7a8' - unihash = 'f37918cc02eb5a520b1aff86faacbc0a38124646' + unihash = '46edb5140d2613049332d0bf3745d9fafec9c559dac8cc61813739a28007fcdf' result = self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') taskhash2 = '3bf6f1e89d26205aec90da04854fbdbf73afe6b4' outhash2 = '77623a549b5b1a31e3732dfa8fe61d7ce5d44b3370f253c5360e136b852967b4' - unihash2 = 'af36b199320e611fbb16f1f277d3ee1d619ca58b' + unihash2 = 'bf6e81926066f770e960f9f777cd088c62bea9addb7745f3e77deaa81a645747' result = self.client.report_unihash(taskhash2, self.METHOD, outhash2, unihash2) self.assertClientGetHash(self.client, taskhash2, unihash2) taskhash3 = 'a1117c1f5a7c9ab2f5a39cc6fe5e6152169d09c0' outhash3 = '7289c414905303700a1117c1f5a7c9ab2f5a39cc6fe5e6152169d09c04f9a53c' - unihash3 = '905303700a1117c1f5a7c9ab2f5a39cc6fe5e615' + unihash3 = 'f5966c8588a69948a89131d3aa63b914c5a63a8cb4820cb1c58694135ba77f0b' result = self.client.report_unihash(taskhash3, self.METHOD, outhash3, unihash3) self.assertClientGetHash(self.client, taskhash3, unihash3) @@ -1014,14 +1071,14 @@ class HashEquivalenceCommonTests(object): def test_gc_switch_mark(self): taskhash = '53b8dce672cb6d0c73170be43f540460bfc347b4' outhash = '5a9cb1649625f0bf41fc7791b635cd9c2d7118c7f021ba87dcd03f72b67ce7a8' - unihash = 'f37918cc02eb5a520b1aff86faacbc0a38124646' + unihash = '46edb5140d2613049332d0bf3745d9fafec9c559dac8cc61813739a28007fcdf' result = self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') taskhash2 = '3bf6f1e89d26205aec90da04854fbdbf73afe6b4' outhash2 = '77623a549b5b1a31e3732dfa8fe61d7ce5d44b3370f253c5360e136b852967b4' - unihash2 = 'af36b199320e611fbb16f1f277d3ee1d619ca58b' + unihash2 = 'bf6e81926066f770e960f9f777cd088c62bea9addb7745f3e77deaa81a645747' result = self.client.report_unihash(taskhash2, self.METHOD, outhash2, unihash2) self.assertClientGetHash(self.client, taskhash2, unihash2) @@ -1059,14 +1116,14 @@ class HashEquivalenceCommonTests(object): def test_gc_switch_sweep_mark(self): taskhash = '53b8dce672cb6d0c73170be43f540460bfc347b4' outhash = '5a9cb1649625f0bf41fc7791b635cd9c2d7118c7f021ba87dcd03f72b67ce7a8' - unihash = 'f37918cc02eb5a520b1aff86faacbc0a38124646' + unihash = '46edb5140d2613049332d0bf3745d9fafec9c559dac8cc61813739a28007fcdf' result = self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') taskhash2 = '3bf6f1e89d26205aec90da04854fbdbf73afe6b4' outhash2 = '77623a549b5b1a31e3732dfa8fe61d7ce5d44b3370f253c5360e136b852967b4' - unihash2 = 'af36b199320e611fbb16f1f277d3ee1d619ca58b' + unihash2 = 'bf6e81926066f770e960f9f777cd088c62bea9addb7745f3e77deaa81a645747' result = self.client.report_unihash(taskhash2, self.METHOD, outhash2, unihash2) self.assertClientGetHash(self.client, taskhash2, unihash2) @@ -1089,7 +1146,7 @@ class HashEquivalenceCommonTests(object): def test_gc_new_hashes(self): taskhash = '53b8dce672cb6d0c73170be43f540460bfc347b4' outhash = '5a9cb1649625f0bf41fc7791b635cd9c2d7118c7f021ba87dcd03f72b67ce7a8' - unihash = 'f37918cc02eb5a520b1aff86faacbc0a38124646' + unihash = '46edb5140d2613049332d0bf3745d9fafec9c559dac8cc61813739a28007fcdf' result = self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') @@ -1106,7 +1163,7 @@ class HashEquivalenceCommonTests(object): taskhash2 = '3bf6f1e89d26205aec90da04854fbdbf73afe6b4' outhash2 = '77623a549b5b1a31e3732dfa8fe61d7ce5d44b3370f253c5360e136b852967b4' - unihash2 = 'af36b199320e611fbb16f1f277d3ee1d619ca58b' + unihash2 = 'bf6e81926066f770e960f9f777cd088c62bea9addb7745f3e77deaa81a645747' result = self.client.report_unihash(taskhash2, self.METHOD, outhash2, unihash2) self.assertClientGetHash(self.client, taskhash2, unihash2) @@ -1405,14 +1462,14 @@ class TestHashEquivalenceClient(HashEquivalenceTestSetup, unittest.TestCase): def test_gc(self): taskhash = '53b8dce672cb6d0c73170be43f540460bfc347b4' outhash = '5a9cb1649625f0bf41fc7791b635cd9c2d7118c7f021ba87dcd03f72b67ce7a8' - unihash = 'f37918cc02eb5a520b1aff86faacbc0a38124646' + unihash = '46edb5140d2613049332d0bf3745d9fafec9c559dac8cc61813739a28007fcdf' result = self.client.report_unihash(taskhash, self.METHOD, outhash, unihash) self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash') taskhash2 = '3bf6f1e89d26205aec90da04854fbdbf73afe6b4' outhash2 = '77623a549b5b1a31e3732dfa8fe61d7ce5d44b3370f253c5360e136b852967b4' - unihash2 = 'af36b199320e611fbb16f1f277d3ee1d619ca58b' + unihash2 = 'bf6e81926066f770e960f9f777cd088c62bea9addb7745f3e77deaa81a645747' result = self.client.report_unihash(taskhash2, self.METHOD, outhash2, unihash2) self.assertClientGetHash(self.client, taskhash2, unihash2) @@ -1455,7 +1512,7 @@ class TestHashEquivalenceUnixServerLongPath(HashEquivalenceTestSetup, unittest.T # Simple test that hashes can be created taskhash = '35788efcb8dfb0a02659d81cf2bfd695fb30faf9' outhash = '2765d4a5884be49b28601445c2760c5f21e7e5c0ee2b7e3fce98fd7e5970796f' - unihash = 'f46d3fbb439bd9b921095da657a4de906510d2cd' + unihash = 'a69ec97f5af2e21e1a1f9cc8896965515d5559425666f734e245a3d40cee33d9' self.assertClientGetHash(self.client, taskhash, None) @@ -1542,4 +1599,3 @@ class TestHashEquivalenceExternalServer(HashEquivalenceTestSetup, HashEquivalenc def test_auth_get_all_users(self): self.skipTest("Cannot test all users with external server") - From patchwork Fri Jun 5 04:58:26 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anders Heimer X-Patchwork-Id: 89334 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45C6ACD6E5D for ; Fri, 5 Jun 2026 04:58:39 +0000 (UTC) Received: from DUZPR83CU001.outbound.protection.outlook.com (DUZPR83CU001.outbound.protection.outlook.com [52.101.66.29]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.1519.1780635511989931224 for ; Thu, 04 Jun 2026 21:58:34 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@est.tech header.s=selector1 header.b=Vf1YHNVb; spf=pass (domain: est.tech, ip: 52.101.66.29, mailfrom: anders.heimer@est.tech) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rYOHMaJFsYq3A6nn8XIysVZPE+YP1sBmBEJH6DsH6RVhdqKFvGfR8Nzsp/IHe/Lc2zcKOo3KlKPJuyxJbH6B0a2tms7OlsP8zc6cw+mJ3nloMYhPjD7dxhVMrUtLHM/BKOqOdfMufJlaEhAxiZ9whE+G/uLJ7bjGbMtAMnDem0lUGLYw2OBR28AkRI8d/XIA5GRcAF8fJ0Sc1oNdcnNMmK60pw4UM1jnmkM+ImAE+9LIhNOaXvCqpO4U5x2TDv0klW+xWeg8zOBjDukJoJbHePs9JCxbfus1vlf4YcqF5K8CbEnFYPXpFdD8AFnOYMjGM+WM6ZGXmohZshbCBMSBxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1tg91+QzR+Cd4Fgk2E6z/stnOD0Fqg3KXJbiU46psJg=; b=Q2Gu2gDMDY1zQcTfUq0rECvxY5ctmr9uv3W+/2SnkFUCQf8wktNmn4GhTePQGZypT4IkcMoet5lrefx3D0WZVwEhYBEpIUU62JB3AAgsTr/dbaO+tNk8QAkHeGZpywTBNehQTZOFVFFzxWMO+QoeJ24YjQ+4sAtZT4ui/iiJZBKjKdoLab94aPwlDX28qQAafOoSdY9kZ/2J51BPYW8E0UW2tFLvRG2F9jmxHbBNyhQym5Dx7XCbZKj+6+PVnDu0b5dwvx7DHxER1vLUBRASU4x7fd7RInYkWcxMUi34ErWieWoxfojm7hqQQq+BmmbeGB2F41Bt8eAAoF/cIxo2KA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=est.tech; dmarc=pass action=none header.from=est.tech; dkim=pass header.d=est.tech; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=est.tech; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1tg91+QzR+Cd4Fgk2E6z/stnOD0Fqg3KXJbiU46psJg=; b=Vf1YHNVbYaLZqoevLQxSZCJHIgc3zByegikCMq7LEGBO3SPea2rbTaW+wRmxGTx7r/PYdwU4wQkNduj7j/t6ZKQm+9Db/UB6lwDhzvaxB+Qg+m7KvPJ5CSFOqCtvbpCyQjoYXo1E4HmmebLEqtu2tqkY3GLay5jF+fcr2wr0su3+X536wrUCMoYz9mK/LprMlU4rhtm5VauUqQ3rOVQuuMU/UJAWsKVQPdjJDLdGDzOdFr4R6Dc210fhJeYvgA6oP0h56SffLHKSRFz0X+wlbeT8qe9mPr+UILP5Dmw90lc/Kz/2ge0JwZqfXicowmWU+9dueNFyrh2NAowkMc7Phg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=est.tech; Received: from DB9P189MB1641.EURP189.PROD.OUTLOOK.COM (2603:10a6:10:2ac::9) by AM0P189MB0785.EURP189.PROD.OUTLOOK.COM (2603:10a6:208:19f::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.8; Fri, 5 Jun 2026 04:58:30 +0000 Received: from DB9P189MB1641.EURP189.PROD.OUTLOOK.COM ([fe80::90da:b700:f102:5c82]) by DB9P189MB1641.EURP189.PROD.OUTLOOK.COM ([fe80::90da:b700:f102:5c82%6]) with mapi id 15.21.0092.007; Fri, 5 Jun 2026 04:58:30 +0000 From: Anders Heimer To: bitbake-devel@lists.openembedded.org Subject: [PATCH 2/2] hashserv: validate unihash values Date: Fri, 5 Jun 2026 06:58:26 +0200 Message-ID: <20260605045826.2052909-3-anders.heimer@est.tech> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260605045826.2052909-1-anders.heimer@est.tech> References: <20260605045826.2052909-1-anders.heimer@est.tech> X-ClientProxiedBy: DB9PR01CA0028.eurprd01.prod.exchangelabs.com (2603:10a6:10:1d8::33) To DB9P189MB1641.EURP189.PROD.OUTLOOK.COM (2603:10a6:10:2ac::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB9P189MB1641:EE_|AM0P189MB0785:EE_ X-MS-Office365-Filtering-Correlation-Id: 9c988abd-d344-43af-0605-08dec2bf166b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016|22082099003|18002099003|6133799003|56012099006|11063799006|5023799004; X-Microsoft-Antispam-Message-Info: 88P0G1KstdKZegQs968vMcXO9SmB3LoLN77NC7o5S8WOROy1lzIuVgrO7satRuhUjDG3k0JTDHvDagcoU+eDgkVQi214Lq+COyoTXWbiPpvDT/As/Wk2DesGv0o/+hTE+opOONdt7PROO2Gdej3yLTejlTLpforM/AOZf3h+fW5zMIzdGP+S0gujFPaIR1iaKP4EXkZGrXEOvR02chkDgqy/kPLyHlQstoaLWLYiwQ2v/U7OGanNvKdeIc+d1+nJ4uW/UdYohv5Pq/TVnpRyHNE9UfUc0e+LuX972ST3k5uaN4GXtzNU+++LZLgg6Xqrke3R8X2ScrXrpQRiwQh5WKuhfUsJOxbb8vqE0pOhRuzUo1mvTxIDV9Epw55DcEi/t5dsiCYrjEo3mMnggAodgrrhjFeRrjZbLA6CN0jbnVKf9mrKASIXfo8TgrCPohwp8+CjOTgd51N9n+JyyucPVlszNR4h2bTQWpRJ6m4ln9CYdG9Vzd+eezsnemSkiDBl0dKXdNrEPw3Xlc7yy+lyr2noC13ad3rtHTZ0L72bSnT8CeQS5cpA4p+D2n74e+bNel3tdT8ASUwa0lpa1/l2EML/1EoDdG/1Ts/aeCX+n34zDaxVgjIrm5MVqjuexOKQOuKrpI4xVstbKcC2mVQOP6oG5ubRGwu0A4xox1qJHgx27jO2NsRosMzt/WByPD/G X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9P189MB1641.EURP189.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016)(22082099003)(18002099003)(6133799003)(56012099006)(11063799006)(5023799004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qOE1zRfV9S1AAo0fQP6LKTeFiyA+16U0AoFXNkjHpTrONUpajHKOxPYsOOem3vmt3+uz6IJtNL/9oO4TrNt2RyPLRM+FbYWmyqxqPknuaerMVnwxbjyZTQSZvdWTTOt+Tsw7kWvb3fz6yt7FA5ap1Fh9Spttb/3rk5X/DaaLwJg5C/6Dt38unkIGJOOXWCFosBrfb2rnZWIROo7eX3yYpOHoGTo38boav6CqkWCxkUSB0ZZemuQPKP5UmAAPphTnO0wJitGHaHSmBf38aIgYtSBEZ+25wbLSzY0IDtNfC5+mKj0IyK58ARIxYr4KFtt8lIZE6/SDgRPEYYF7876k+M+TWg53pKDWAbEgO/y/yFwOvyilRNysUCK6rOQdE8xR5vZrCKFt8j1mxoHTTLeSqwViXVV0zCBUYzBxZs2f9rvTrEY+AQyygY6ajXXO+wc347xZ63nYz2kv9f0kS/mZpX573Jr46amfDEooXPHfW/VnEHSMY9gMU3IYJkwFXXjfP1Dq+O0zWuge9QtA76MAhDwU9dKGL/qYdr+N1WzgYyWD2Y9eSCWroSLKvyyfKHfF2Z5Sd6lLwTfCve2mVUMcYe/gsT1pu462XAZi+Nyr1XvJRubcLfdoxM3aEWkgkSo8c7tR5KXCprXNat/fno9tHiOEY1hPgaiQeYDXk2zsC3yz8VzaMAI+mKpv9PRfhXio54ax42AzQk/jmDgJ0zQBPmxAW6h0BSnrcAyOIBkeT7+o0gEmNIeT15n3lnyKiw6I3REF85OVk9G3hiV+FN+LLVARr18eOu2lnyXV361WXNJfp39L8E2Xm094PsNTo/POL7qOFqItYC/EdzE/wk2N/n6F+YOg+k6+aKo78LAXkIqQ7hx7CRuDdSDpKNPF9/7IAVk6ONOjwEaaUqYinQslGbaVRMTECGSjAU2oKqFGVHEj6wzkXvHiebznWW448/PDmlW+B+KMDvqzXFqH7P6/Set9nR6bagm5XjAWS2IRmAKCSvQxPmmuPFV6ZUys06cZN/vDU/uryrDbzkU9eq3vZ5IY/ea+fjVLCukuHmrpf+tSVkot28CivDxvTV92Ikn94BLcjSOje8mA4SA+Q96+/Oe54oVp0zFTAj/wzXUUQ+fhH80bed3NqoEg0oTjX1Xkc5de3VHpWi+NHIkQqOYeQ/RiWZDYWeWE4b/3B7LmHYBsNUNfLZuTglt9PqsEdjf6IOtxbhw6zTCAF9KwjJssF+u/4vOu765ZEJXoGcfQKViL1e8D4+x/y3SdlRQ3lw8z+jUtzR82u4BWtJC2W8OgJwmAH0OnDioG357aPTN3e07DuovO4zREaRDTJ1Rgd2hg3SNVhUSKryEDrY4afRo2TaV+J+NzdoW+B+WuiWvqBw9tBW7nCZaLNGnPWpKkAvHa5sNrG4JnjwSd1wi9npy0fnFKdfoaQbT55jR9KSLaG1csOcLCmMVYXg8hvUFKj69WFaKK2l0tBu6GkpfDsQCS1vA65+u8YWZYY+heOfwA5trrkRNK3nDY44WoNW35yexNlWJJ4KtdEjiVx/HdL5B0j4OO5IK6O4AoLL5gYR4l1IngYNDzql/2metWjMrWLzEaoYOQdYmoF3R2n/2ZM++tTEQSwEhJQ8PyoyYRnaTCM0NhVJGFIPEyL8ydA+4QL6FOozVv/7SqCsxFA8luSouD3sbLq0mJKbBGG/IULUX5TcgS3ujGMFI4cYbwOLJPy4r9OCYI7tdyeiBhsevjPCTy8Q== X-OriginatorOrg: est.tech X-MS-Exchange-CrossTenant-Network-Message-Id: 9c988abd-d344-43af-0605-08dec2bf166b X-MS-Exchange-CrossTenant-AuthSource: DB9P189MB1641.EURP189.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2026 04:58:30.4866 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: d2585e63-66b9-44b6-a76e-4f4b217d97fd X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cpOPwg9mFvr0OfJDxQ4G4zWyGvcs9nVbJ9dm626TRs6KXCg9oE2j1pNCwCx7q4vRVGoH8TX136t3VzCvPmHFiQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0P189MB0785 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 05 Jun 2026 04:58:39 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19614 Signed-off-by: Anders Heimer --- lib/bb/siggen.py | 9 +++++++- lib/bb/tests/siggen.py | 47 ++++++++++++++++++++++++++++++++++++++++ lib/hashserv/__init__.py | 7 ++++++ lib/hashserv/server.py | 27 ++++++++++++++++++----- lib/hashserv/tests.py | 30 +++++++++++++++++++++++++ 5 files changed, 114 insertions(+), 6 deletions(-) diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py index 985fa7e4c..3a203676e 100644 --- a/lib/bb/siggen.py +++ b/lib/bb/siggen.py @@ -43,6 +43,10 @@ def check_siggen_version(siggen): if siggen.find_siginfo_version < siggen.find_siginfo_minversion: bb.fatal("Siggen from metadata (OE-Core?) is too old, please update it (%s vs %s)" % (siggen.find_siginfo_version, siggen.find_siginfo_minversion)) +def check_hashserv_unihash(unihash): + if not hashserv.is_valid_unihash(unihash): + bb.fatal("Hash Equivalence Server returned invalid unihash") + class SetEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, set) or isinstance(obj, frozenset): @@ -729,6 +733,7 @@ class SignatureGeneratorUniHashMixIn(object): if unihashes and unihashes[idx]: unihash = unihashes[idx] + check_hashserv_unihash(unihash) # A unique hash equal to the taskhash is not very interesting, # so it is reported it at debug level 2. If they differ, that # is much more interesting, so it is reported at debug level 1 @@ -747,7 +752,7 @@ class SignatureGeneratorUniHashMixIn(object): import importlib taskhash = d.getVar('BB_TASKHASH') - unihash = d.getVar('BB_UNIHASH') + unihash = d.getVar('BB_UNIHASH', expand=False) report_taskdata = d.getVar('SSTATE_HASHEQUIV_REPORT_TASKDATA') == '1' tempdir = d.getVar('T') mcfn = d.getVar('BB_FILENAME') @@ -809,6 +814,7 @@ class SignatureGeneratorUniHashMixIn(object): data = client.report_unihash(taskhash, method, outhash, unihash, extra_data) new_unihash = data['unihash'] + check_hashserv_unihash(new_unihash) if new_unihash != unihash: hashequiv_logger.debug('Task %s unihash changed %s -> %s by server %s' % (taskhash, unihash, new_unihash, self.server)) @@ -848,6 +854,7 @@ class SignatureGeneratorUniHashMixIn(object): return False finalunihash = data['unihash'] + check_hashserv_unihash(finalunihash) if finalunihash == current_unihash: hashequiv_logger.verbose('Task %s unihash %s unchanged by server' % (tid, finalunihash)) diff --git a/lib/bb/tests/siggen.py b/lib/bb/tests/siggen.py index 0dc67e6cc..eb07cc920 100644 --- a/lib/bb/tests/siggen.py +++ b/lib/bb/tests/siggen.py @@ -9,7 +9,9 @@ import unittest import logging import bb +import bb.data import time +from contextlib import contextmanager logger = logging.getLogger('BitBake.TestSiggen') @@ -26,3 +28,48 @@ class SiggenTest(unittest.TestCase): for t in tests: self.assertEqual(bb.siggen.build_pnid(*t), tests[t]) + def test_get_unihashes_rejects_invalid_hashserv_unihash(self): + class TestClient: + def get_unihash_batch(self, query): + list(query) + return ["${@os.system('true')}"] + + class TestSiggen(bb.siggen.SignatureGeneratorUniHashMixIn): + def __init__(self): + self.server = "test-server" + self.method = "test-method" + self.extramethod = {} + self.taskhash = {"test.bb:do_compile": "a" * 64} + self.unihash = {} + self.unitaskhashes = {} + self.tidtopn = {} + self.setscenetasks = set() + + @contextmanager + def client(self): + yield TestClient() + + siggen = TestSiggen() + + with self.assertRaises(bb.BBHandledException): + siggen.get_unihashes(["test.bb:do_compile"]) + + self.assertEqual(siggen.unihash, {}) + self.assertEqual(siggen.unitaskhashes, {}) + + def test_report_unihash_reads_bb_unihash_without_expansion(self): + class TestSiggen(bb.siggen.SignatureGeneratorUniHashMixIn): + def __init__(self): + self.setscenetasks = set() + self.taskhash = {"test.bb:do_compile": "b" * 64} + + d = bb.data.init() + d.setVar("BB_TASKHASH", "a" * 64) + d.setVar("BB_UNIHASH", "${@d.setVar('EXPANDED_UNIHASH', '1') or 'bad'}") + d.setVar("SSTATE_HASHEQUIV_REPORT_TASKDATA", "0") + d.setVar("T", "/tmp") + d.setVar("BB_FILENAME", "test.bb") + + TestSiggen().report_unihash(".", "compile", d) + + self.assertIsNone(d.getVar("EXPANDED_UNIHASH")) diff --git a/lib/hashserv/__init__.py b/lib/hashserv/__init__.py index ac891e017..ba8e0acce 100644 --- a/lib/hashserv/__init__.py +++ b/lib/hashserv/__init__.py @@ -7,12 +7,19 @@ import asyncio from contextlib import closing import itertools import json +import re from collections import namedtuple from urllib.parse import urlparse from bb.asyncrpc.client import parse_address, ADDR_TYPE_UNIX, ADDR_TYPE_WS User = namedtuple("User", ("username", "permissions")) +UNIHASH_REGEX = re.compile(r"^[0-9a-f]{64}$") + + +def is_valid_unihash(value): + return isinstance(value, str) and UNIHASH_REGEX.fullmatch(value) is not None + def create_server( addr, diff --git a/lib/hashserv/server.py b/lib/hashserv/server.py index 58f95c7bc..3ff434785 100644 --- a/lib/hashserv/server.py +++ b/lib/hashserv/server.py @@ -13,6 +13,7 @@ import base64 import json import hashlib from . import create_async_client +from . import is_valid_unihash import bb.asyncrpc logger = logging.getLogger("hashserv.server") @@ -173,6 +174,11 @@ def hash_token(algo, salt, token): return ":".join([algo, salt, h.hexdigest()]) +def validate_unihash(value): + if not is_valid_unihash(value): + raise bb.asyncrpc.InvokeError("Invalid unihash") + + def permissions(*permissions, allow_anon=True, allow_self_service=False): """ Function decorator that can be used to decorate an RPC function call and @@ -345,7 +351,7 @@ class ServerClient(bb.asyncrpc.AsyncServerConnection): d = {k: row[k] for k in row.keys()} elif self.upstream_client is not None: d = await self.upstream_client.get_taskhash(method, taskhash) - await self.db.insert_unihash(d["method"], d["taskhash"], d["unihash"]) + await self.insert_unihash(d["method"], d["taskhash"], d["unihash"]) return d @@ -377,9 +383,13 @@ class ServerClient(bb.asyncrpc.AsyncServerConnection): if data is None: return - await self.db.insert_unihash(data["method"], data["taskhash"], data["unihash"]) + await self.insert_unihash(data["method"], data["taskhash"], data["unihash"]) await self.db.insert_outhash(data) + async def insert_unihash(self, method, taskhash, unihash): + validate_unihash(unihash) + return await self.db.insert_unihash(method, taskhash, unihash) + async def _stream_handler(self, handler): await self.socket.send_message("ok") @@ -467,6 +477,8 @@ class ServerClient(bb.asyncrpc.AsyncServerConnection): # report is made inside the function @permissions(READ_PERM) async def handle_report(self, data): + validate_unihash(data.get("unihash")) + if self.server.read_only or not self.user_has_permissions(REPORT_PERM): return await self.report_readonly(data) @@ -509,7 +521,7 @@ class ServerClient(bb.asyncrpc.AsyncServerConnection): if upstream_data is not None: unihash = upstream_data["unihash"] - await self.db.insert_unihash(data["method"], data["taskhash"], unihash) + await self.insert_unihash(data["method"], data["taskhash"], unihash) unihash_data = await self.get_unihash(data["method"], data["taskhash"]) if unihash_data is not None: @@ -525,7 +537,9 @@ class ServerClient(bb.asyncrpc.AsyncServerConnection): @permissions(READ_PERM, REPORT_PERM) async def handle_equivreport(self, data): - await self.db.insert_unihash(data["method"], data["taskhash"], data["unihash"]) + validate_unihash(data.get("unihash")) + + await self.insert_unihash(data["method"], data["taskhash"], data["unihash"]) # Fetch the unihash that will be reported for the taskhash. If the # unihash matches, it means this row was inserted (or the mapping @@ -888,7 +902,10 @@ class Server(bb.asyncrpc.AsyncServer): method, taskhash = item d = await client.get_taskhash(method, taskhash) if d is not None: - await db.insert_unihash(d["method"], d["taskhash"], d["unihash"]) + if is_valid_unihash(d.get("unihash")): + await db.insert_unihash(d["method"], d["taskhash"], d["unihash"]) + else: + self.logger.warning("Upstream server returned invalid unihash") self.backfill_queue.task_done() def start(self): diff --git a/lib/hashserv/tests.py b/lib/hashserv/tests.py index 83ce0c5ca..c2ed1035a 100644 --- a/lib/hashserv/tests.py +++ b/lib/hashserv/tests.py @@ -292,6 +292,36 @@ class HashEquivalenceCommonTests(object): self.assertEqual(result_outhash['outhash'], outhash) self.assertEqual(result_outhash['outhash_siginfo'], siginfo) + def test_report_rejects_invalid_unihash(self): + taskhash = '68a9206490b2321bb033fb3eab013a4ec62c41f9' + outhash = 'bf5f2efaf1ca351f3b4c3d079363540ab48f7c58db3d23cfbb069cf4ff1ea8f7' + invalid_unihashes = ( + "${@os.system('true')}", + 'a' * 63, + 'a' * 65, + 'A' * 64, + None, + ) + + for unihash in invalid_unihashes: + with self.subTest(unihash=unihash): + with self.start_client(self.server_address) as client: + with self.assertRaises(InvokeError) as context: + client.report_unihash(taskhash, self.METHOD, outhash, unihash) + + self.assertEqual(str(context.exception), "Invalid unihash") + + self.assertClientGetHash(self.client, taskhash, None) + + def test_equivreport_rejects_invalid_unihash(self): + taskhash = 'ae6339531895ddf5b67e663e6a374ad8ec71d81c' + + with self.assertRaises(InvokeError) as context: + self.client.report_unihash_equiv(taskhash, self.METHOD, "${@os.system('true')}") + + self.assertEqual(str(context.exception), "Invalid unihash") + self.assertClientGetHash(self.start_client(self.server_address), taskhash, None) + def test_stress(self): def query_server(failures): client = Client(self.server_address)