From patchwork Mon Sep 21 00:41:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 98752 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90C20C982E1 for ; Mon, 21 Sep 2026 00:46:02 +0000 (UTC) Received: from mail-pj2-f12.google.com (mail-pj2-f12.google.com [74.125.227.140]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.40006.1789951554935211090 for ; Sun, 20 Sep 2026 17:45:55 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@konsulko.com header.s=google header.b=hvD8oyD+; spf=pass (domain: konsulko.com, ip: 74.125.227.140, mailfrom: tim.orling@konsulko.com) Received: by mail-pj2-f12.google.com with SMTP id d9443c01a7336-2d8fb334ddcso19896065ad.0 for ; Sun, 20 Sep 2026 17:45:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; t=1789951554; x=1790556354; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=HJiGDXwtCReAkKX0mbvc9oUfBAMKl9IAMAu5Hv73O0E=; b=hvD8oyD+hGA+99x5etj1DMGnwaRUdjg61ysJQu35jS0H+Ri94uDcVTRbZLec8szo1J MsJ2gk9/GISAGSADEmEF9sed8y56NdLnd9olu79FVIDsoHlu19AlCu4Yn1cMz+UOfKlo 95M038TMTE53dC3FzzIQPfEnaWFx4YshrQ9Tw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789951554; x=1790556354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=HJiGDXwtCReAkKX0mbvc9oUfBAMKl9IAMAu5Hv73O0E=; b=g0Nak6d1wU1otOUohN377odfi/3qVp/OQr0R5pfrR1qnjyyQljllOW70ogvi98l9Yl RlX2e3L/wGu9l+g24jtZLNYUA0/Ig2WSVQ9Ny4he8/sPO7ADG3m1ya857rqilmzda3H2 xtZokm2HwN0nP8UV66LqXeYva+HFNZeT3L5sfu1ZjyOzJtQ9jRsAeRjkAPaPlqjWbCFv K3aTsJhX8VLtxshCwYkRFg/MbMIwCFTSxIK3p9efRORf7hANzLr97OJJcoFBaUcGLKgb ZFiUmpBQj1xhVH7YeX2sIpRenVst0WdFOqm51wfMH4wfb4m+1uWoMKY9ul1zNhjPkYFE WCoA== X-Gm-Message-State: AFuF++mK+oKpZC8GkzgqIycgNNrZgrQ3vwJgq10tVkrX8G8CrB7Ow5T6 eUBzAgEtC0uychsqwD6RB5geyMQVHFCmpyjxnNdqILdL7Y3N/DLsTnzQAp/zarUFczM+4oFjUnv 5xcty X-Gm-Gg: AYBFou3RlF3lrj8g5Qs1+ycjKsxJf/TU4Qw6RSeRZ4aKf0EjMxw6+BdgirjPvgVJPvf RD8PU5IwPMs0FAKpx2SDbqPuEQCI4V24PWxDNGbGmm8yTZ9Hadvq0c+VM2HWBpi7ZDPiN177FjL AaUSx/z28kt01E0xHbgsFxj9TPpTIW10KrODPwjHtL9pmqlFH3hpbiNuyS5qzRPqFl4uLUE6e+G g4X5IUVvW7YuRb3LXxQNoFZP0sueG4GVcYOPfbcHws+UhjyvMdFf2PLc7mifAXoXRCmX25ql+zT uENw+GzsQUi38l+YwwwLHCtk+lIS/kcyCBtGp+17gwYz+TlmI21d2/0wn+QQ1NeSHeAxfSJW7rO B5QmewlwBK7tiiQIfZsb+VnadHS2AI5RCduSOC1l/hqaTtxdU7PK1lxEo+o5DeO+dtDJOcri/+L S6JTt09UzfvjT7hNmVoeP5B+BbIK032hhT1W+uOxcyz5lpISmf4UVqY12ZCotDbHO6RQM/NlBhe uSeUGa87hT78A== X-Received: by 2002:a17:902:ce07:b0:2da:f1b1:56c4 with SMTP id d9443c01a7336-2ddb1ab3254mr148186015ad.3.1789951554199; Sun, 20 Sep 2026 17:45:54 -0700 (PDT) Received: from thetis.home.local ([2606:c800:6024:2000:7821:4c5:938a:e12]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2ddc1803813sm24512385ad.82.2026.09.20.17.45.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Sep 2026 17:45:53 -0700 (PDT) From: tim.orling@konsulko.com To: yocto-patches@lists.yoctoproject.org Cc: Tim Orling Subject: [yocto-autobuilder-helper][PATCH 1/4] run-push-containers: Set COSIGN_YES to avoid interactive prompt Date: Sun, 20 Sep 2026 17:41:15 -0700 Message-ID: <20260921004540.3718904-2-tim.orling@konsulko.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260921004540.3718904-1-tim.orling@konsulko.com> References: <20260921004540.3718904-1-tim.orling@konsulko.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Sep 2026 00:46:02 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/4914 From: Tim Orling cosign 3.x asks for consent before uploading to the Rekor transparency log, which the autobuilder cannot answer: By typing 'y', you attest that (1) you are not submitting the personal data of any other person; and (2) you understand and agree to the statement and the Agreement terms at the URLs listed above. Are you sure you would like to continue? [y/N] Error: signing [...]: recursively signing: signing digest: user declined the prompt Export COSIGN_YES=true alongside COSIGN_PASSWORD so both 'cosign sign' and 'cosign attest' run non-interactively. AI-Generated: Claude Opus 5 Signed-off-by: Tim Orling --- scripts/run-push-containers | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/run-push-containers b/scripts/run-push-containers index 5dd35af..3ed43ae 100755 --- a/scripts/run-push-containers +++ b/scripts/run-push-containers @@ -164,6 +164,11 @@ script = [ # The real value, if the key is encrypted, must be present in the build # environment. # +# COSIGN_YES answers the transparency-log consent prompt ("Are you sure you +# would like to continue? [y/N]") that cosign shows before uploading to Rekor. +# Without it the non-interactive autobuilder gets "user declined the prompt" +# and the step fails. It covers both 'sign' and 'attest'. +# # sign_image() signs by digest ($2) so cosign does not warn about signing a # mutable tag, while still pinning the index a consumer verifies. The digest # MUST be the digest the tag resolves to: @@ -196,6 +201,7 @@ sys.stdout.write(base64.b64decode(t).decode() if t else "")' "%s" "$_host") done""" % (" ".join(registries), auth_config) script += [ "export COSIGN_PASSWORD=\"${COSIGN_PASSWORD:-}\"", + "export COSIGN_YES=true", "_COSIGN_READY=0", "prepare_cosign() {", " if [ \"$_COSIGN_READY\" = 1 ]; then return 0; fi", From patchwork Mon Sep 21 00:41:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 98750 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEC6CC982EF for ; Mon, 21 Sep 2026 00:46:02 +0000 (UTC) Received: from mail-pz2-f42.google.com (mail-pz2-f42.google.com [74.125.228.42]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.40007.1789951557567213434 for ; Sun, 20 Sep 2026 17:45:57 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@konsulko.com header.s=google header.b=eZr8DXoz; spf=pass (domain: konsulko.com, ip: 74.125.228.42, mailfrom: tim.orling@konsulko.com) Received: by mail-pz2-f42.google.com with SMTP id 41be03b00d2f7-cc4c3304784so2105479a12.3 for ; Sun, 20 Sep 2026 17:45:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; t=1789951557; x=1790556357; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=DOfRFZ1pAeOGLZ2vuY2FyCFNOV7rdnmpSwtHL8/EtTw=; b=eZr8DXoznJL8DSe3NE4yI9uHFzVl6FW/X/ehP2heFkYSQwhtWsvMeA70YrptYoUDi5 Jjlv8iUbrnS++nmr30U2GWYVYnxssRZBsAqFWF6OLhsXJTJA5dy0rGmCcN/wOL6rEyn9 vqpw4aroyIujt+kiKwcID275MAjJKP4CNGm7s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789951557; x=1790556357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=DOfRFZ1pAeOGLZ2vuY2FyCFNOV7rdnmpSwtHL8/EtTw=; b=LL/1bQv0XpP/NoG6aqdgg5YCwOAUZxrfpZKLYHkqvtU0kJN3w9JLZOaNAyNADqUPoq eRq7D3o/osX/LSP/2ZxzRfoDkq9hGvt1ufls0mWhJzYZzYcaCtf4Z+xluednc8n60K2B +IgsIaKelceYwlGRKKm8/NBEptnUPdrPi255TtkLfbUPD8AtgYH135sEhCN4v7lPdgqq 3f5ahxUvUpzYJowVM8/o4TZsRt1orp90pgye1t9XT32TANAZtCU4/FxlhvWb7yShdVJv ll3n7LOp7JxqYhAn2SVm+fxDZ6hribeVnJbfkP+gaxbG9sDVm54D1Cx/fQ8waLeulZ6/ x8SQ== X-Gm-Message-State: AFuF++ng44lUhwMsjHKRBS8ma9AL1lB2ZI6igi1dSdxjydXzwiIvFdiS XuYYX4EYgMUTgcioiIEVqoEfpGNZZGRBU2+Qy391POEqqU2o93gn2QnDHFHPWCegrTDhROiyoZh LM2r1 X-Gm-Gg: AYBFou2L2LDnAL25YC98Twt730VYrHT99KWZSiaFSRaLM2lp0OYUReHcg2H8rPrVZY6 s1DhvOykx8fuBk3RqTRCWveIEGv4rNeXL/YVYqaawIAEJegj1Pfm1z+VFnIwwnpFk6dC+NOAYYq n633MAhpvdE5azcPR2XsMYW//aPeYc+Dhc5uuyWyWZZi8id2LRoxDr1tMpL6gDYWCentMGufyXn lafjPXl/6u//YGGpW+QvbSy+nxGlCRYnzypgeqarhkt4xppanhAmI1Yrl+y9hRH6XWm9P8A8fUS 1zaNHph/JCp8v8SON12WGz4PT1ZrT+Wdgic3mlHyzK3KxY+BYTkUfwmvZCBevfPNnEDGiaiKYnf BS0TOEFkQ/h3bI6A/8Z8dffrWif60mhytmqklnf5aYEcHsHrrtE2FFhXKukzfpr8mMVr11V0R8l HklvHSOuwDTg3tXR8nkU6k+XIHmzHwgtfKUlYDxYIgdkpaZPUbKwZfVPoIzehVWfz6e2cl87uKo QNuTzYRVj+TIz/zcIEESnQiog== X-Received: by 2002:a17:902:a9cb:b0:2dd:c053:e668 with SMTP id d9443c01a7336-2ddc053e68amr51933105ad.46.1789951556763; Sun, 20 Sep 2026 17:45:56 -0700 (PDT) Received: from thetis.home.local ([2606:c800:6024:2000:7821:4c5:938a:e12]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2ddc1803813sm24512385ad.82.2026.09.20.17.45.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Sep 2026 17:45:56 -0700 (PDT) From: tim.orling@konsulko.com To: yocto-patches@lists.yoctoproject.org Cc: Tim Orling Subject: [yocto-autobuilder-helper][PATCH 2/4] run-push-containers: Declare SPDX 3 predicate type for attestations Date: Sun, 20 Sep 2026 17:41:16 -0700 Message-ID: <20260921004540.3718904-3-tim.orling@konsulko.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260921004540.3718904-1-tim.orling@konsulko.com> References: <20260921004540.3718904-1-tim.orling@konsulko.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Sep 2026 00:46:02 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/4915 From: Tim Orling cosign's 'spdxjson' alias hardcodes the SPDX 2.x predicate type https://spdx.dev/Document, but create_image_sbom_spdx emits SPDX 3.0.1. Pass the in-toto SPDX 3 URI https://spdx.dev/Document/v3 explicitly on both attest and verify-attestation; for a non-alias type cosign embeds the JSON predicate verbatim, which is what the JSON-LD document needs. AI-Generated: Claude Opus 5 Signed-off-by: Tim Orling --- scripts/run-push-containers | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/scripts/run-push-containers b/scripts/run-push-containers index 3ed43ae..fd606c0 100755 --- a/scripts/run-push-containers +++ b/scripts/run-push-containers @@ -19,7 +19,7 @@ # successfully pushed image is signed with # cosign (otherwise signing is skipped). Also # gates SPDX SBOM attestation: each pushed image -# gets a 'cosign attest --type spdxjson' +# gets an in-toto SPDX 3.x ('cosign attest') # attestation of its SPDX SBOM (per-arch on the # multi-arch path, top manifest on single-arch). # CONTAINER_COSIGN_PUB - cosign public key path; when set, each @@ -69,6 +69,10 @@ cosign_key = utils.getconfigvar("CONTAINER_COSIGN_KEY", ourconfig, args.target, # Public key for verifying attestations; when unset, attestations are made but # not verified. cosign_pub = utils.getconfigvar("CONTAINER_COSIGN_PUB", ourconfig, args.target, args.stepnum) +# in-toto predicate type URI for SPDX 3.x documents. Must be identical on +# 'attest' and 'verify-attestation', or verification finds no matching +# attestation. +SPDX_PREDICATE_TYPE = "https://spdx.dev/Document/v3" utils.printheader("Pushing container images %s" % list(container_images.keys())) @@ -269,7 +273,13 @@ sys.stdout.write(base64.b64decode(t).decode() if t else "")' "%s" "$_host") " prepare_cosign", " case \" $_ATTESTED_REFS \" in *\" $1 \"*) return 0 ;; esac", " _ATTESTED_REFS=\"$_ATTESTED_REFS $1\"", - " _att_out=$(oe-run-native cosign-native cosign attest --key %s --type spdxjson --predicate \"$2\" \"$1\" 2>&1) || {" % cosign_key, + # --type is the in-toto predicate type URI, not cosign's 'spdxjson' + # alias: that alias hardcodes https://spdx.dev/Document, which is the + # SPDX 2.x predicate. create_image_sbom_spdx emits SPDX 3.0.1, so we + # pass the v3 URI explicitly. For an unknown (non-alias) type cosign + # parses the predicate as JSON and embeds it verbatim, which is what + # we want for the SPDX 3 JSON-LD document. + " _att_out=$(oe-run-native cosign-native cosign attest --key %s --type %s --predicate \"$2\" \"$1\" 2>&1) || {" % (cosign_key, SPDX_PREDICATE_TYPE), " case \"$_att_out\" in", " *\"already exists\"*|*createLogEntryConflict*) echo \"cosign: attestation for $1 already in transparency log\" ;;", " *) echo \"$_att_out\" >&2; return 1 ;;", @@ -279,7 +289,7 @@ sys.stdout.write(base64.b64decode(t).decode() if t else "")' "%s" "$_host") ] if cosign_pub: script += [ - " oe-run-native cosign-native cosign verify-attestation --key %s --type spdxjson \"$1\" >/dev/null \\" % cosign_pub, + " oe-run-native cosign-native cosign verify-attestation --key %s --type %s \"$1\" >/dev/null \\" % (cosign_pub, SPDX_PREDICATE_TYPE), " && echo \"cosign: verified attestation for $1\" \\", " || { echo \"ERROR: attestation verification failed for $1\" >&2; return 1; }", ] From patchwork Mon Sep 21 00:41:17 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 98749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6998C982DA for ; Mon, 21 Sep 2026 00:46:00 +0000 (UTC) Received: from mail-pj2-f42.google.com (mail-pj2-f42.google.com [74.125.227.170]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.39803.1789951559853327545 for ; Sun, 20 Sep 2026 17:45:59 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@konsulko.com header.s=google header.b=kw9hpMHF; spf=pass (domain: konsulko.com, ip: 74.125.227.170, mailfrom: tim.orling@konsulko.com) Received: by mail-pj2-f42.google.com with SMTP id d9443c01a7336-2df4c9d14b8so432775ad.0 for ; Sun, 20 Sep 2026 17:45:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; t=1789951559; x=1790556359; darn=lists.yoctoproject.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=RmAxzzCY1yLb1VROhCPO4RSJ+hZHk7Vyp0imBoE9Yzs=; b=kw9hpMHFGOue85BOkriqWzLh1JXqByDZFqbkfXflLnAc0d40iTP9K0ZnvADaFPhSTm +HQG2aDJxiizqnYpeDBmqWGCNIXD9TR3GN3wDTTTtmKSvcscnT45QJeW9DDDlBngtd70 NI87zynQ3RMT3mNMb433oxErP2QcUlhyPYti4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789951559; x=1790556359; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=RmAxzzCY1yLb1VROhCPO4RSJ+hZHk7Vyp0imBoE9Yzs=; b=gAGMpFefXlOroLEzwU3My7vjI3ifY1y1/94GCnvjCUbON6MmW2oLT5LuTnz06cH32m L0r5m5HXbTHmisTU149eg7G6lFuhDNuUtanDPf8qNzEHxQoPxo+olgDbCRJDgNPw78aC Xc22k2ZEnGS9ROOYZ7F+Msm6IB+MyphHfaiTduHKMJYzNI+22TaCUsvvPUtCdwiC8Qe6 0S5COcJT+tKBfRK9zYsgRjTygnxj5VdRXi3ANrw7Q4YpHcO0g4O0OwMjWDQExzXp5Z20 7uUP/6KYPc2+HfpvPHJVk1LHObmiMkIRFvtwXDyTa4hJQl3vQcDDuB37l9vas69QCSM7 QCcQ== X-Gm-Message-State: AFuF++mmWNv0wELZdC8WFLyQEXV6FhFpLUCYWpPKMFqA5+H2ir4kuabN S7er0ixNFz8xOYRFxJxbzIFX+xAfYbyorbNu2b+3Hngm+eySyShL8RNC+mKLzzL99y62aqGMSIQ SAk+w X-Gm-Gg: AYBFou31ELjJFAJrECdm5Ny7QUWG8pxS4ZOGjiTQp9Ea+x1o17qx/S8enVeWJvWI+4R eK/Ja4pC3QfDaa0vmxygJbynigmb0HzwWWUfc6zn6klsmFE9mEUysXvih/1rs9qtsUi2zD/pXrp BPmgN9teJumEDT+lhuWJsSII0m3JfUMouN0u/hZeuA7U/PZ9Quz8frLkycVsKaTkl9unD9IUa6M rqgccrReWZ8lDJutMxLTdKVvFCcQZt5WgA1YAsT+UYamvsrdMTeIkjxA6RUF8illHAbQ9isn7Kz C6KLMnm8X6OzqTKjpGOoyQVrbxMOe18wLR7DlppkR5FOP1PFContdi07GQ5RAKhyPtiGB1OtC52 pqHUMYWa1+6B7ZCACLLqYD8JvegazYJ1g22SYh04Wqo7n7IGG6T+sMJzNo6Yv4m/YssDXA597Xe wIqWHMkC9zeQmeT1Hx6Ijs8MUzyyUR3dYmB23lMQdBuFkeao9MTnJOh/sqqnhYeuqac2Gu17Wzw CauyCTC8HLt5A== X-Received: by 2002:a17:903:1b6c:b0:2dd:c053:b9c7 with SMTP id d9443c01a7336-2ddc053ba17mr58692465ad.24.1789951559191; Sun, 20 Sep 2026 17:45:59 -0700 (PDT) Received: from thetis.home.local ([2606:c800:6024:2000:7821:4c5:938a:e12]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2ddc1803813sm24512385ad.82.2026.09.20.17.45.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Sep 2026 17:45:58 -0700 (PDT) From: tim.orling@konsulko.com To: yocto-patches@lists.yoctoproject.org Cc: Tim Orling Subject: [yocto-autobuilder-helper][PATCH 3/4] run-push-containers: Cache native sysroots to cut bitbake startups Date: Sun, 20 Sep 2026 17:41:17 -0700 Message-ID: <20260921004540.3718904-4-tim.orling@konsulko.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260921004540.3718904-1-tim.orling@konsulko.com> References: <20260921004540.3718904-1-tim.orling@konsulko.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Sep 2026 00:46:00 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/4916 From: Tim Orling Every oe-run-native invocation re-runs bitbake-getvar to locate the native sysroot, paying a full cooker startup and metadata parse each time. Resolve it once per tool in prepare_skopeo()/prepare_cosign() and pass OECORE_NATIVE_SYSROOT through the _skopeo()/_cosign() wrappers. For one multiarch container with 4 tags, 1 registry and 2 platforms this drops bitbake startups from 19 to 11. AI-Generated: Claude Opus 5 Signed-off-by: Tim Orling --- scripts/run-push-containers | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/scripts/run-push-containers b/scripts/run-push-containers index fd606c0..702335b 100755 --- a/scripts/run-push-containers +++ b/scripts/run-push-containers @@ -138,10 +138,16 @@ script = [ " %s-$(arch) --config %s memres restart /: just pushed; $2 = its digest (the index # digest from 'skopeo copy --digestfile' on the multi-arch path). When # $2 is empty (single-arch), resolve the lone manifest digest with - # 'skopeo inspect' — correct there since there is no list. oe-run-native - # prints 'Getting sysroot...' to stdout, so grep the digest out. + # 'skopeo inspect' — correct there since there is no list. The digest + # is grepped out of resolve_digest's output (see there). "sign_image() {", " prepare_skopeo", " prepare_cosign", @@ -241,7 +250,7 @@ sys.stdout.write(base64.b64decode(t).decode() if t else "")' "%s" "$_host") # transparency-log conflict ('already exists' / HTTP 409, which also # recurs on rebuilds that reproduce the same digest) as success so # signing stays idempotent instead of aborting the step. - " _sign_out=$(oe-run-native cosign-native cosign sign --recursive --key %s \"$_SIG_REF\" 2>&1) || {" % cosign_key, + " _sign_out=$(_cosign sign --recursive --key %s \"$_SIG_REF\" 2>&1) || {" % cosign_key, " case \"$_sign_out\" in", " *\"already exists\"*|*createLogEntryConflict*) echo \"cosign: $_SIG_REF already in transparency log, treating as signed\" ;;", " *) echo \"$_sign_out\" >&2; return 1 ;;", @@ -279,7 +288,7 @@ sys.stdout.write(base64.b64decode(t).decode() if t else "")' "%s" "$_host") # pass the v3 URI explicitly. For an unknown (non-alias) type cosign # parses the predicate as JSON and embeds it verbatim, which is what # we want for the SPDX 3 JSON-LD document. - " _att_out=$(oe-run-native cosign-native cosign attest --key %s --type %s --predicate \"$2\" \"$1\" 2>&1) || {" % (cosign_key, SPDX_PREDICATE_TYPE), + " _att_out=$(_cosign attest --key %s --type %s --predicate \"$2\" \"$1\" 2>&1) || {" % (cosign_key, SPDX_PREDICATE_TYPE), " case \"$_att_out\" in", " *\"already exists\"*|*createLogEntryConflict*) echo \"cosign: attestation for $1 already in transparency log\" ;;", " *) echo \"$_att_out\" >&2; return 1 ;;", @@ -289,7 +298,7 @@ sys.stdout.write(base64.b64decode(t).decode() if t else "")' "%s" "$_host") ] if cosign_pub: script += [ - " oe-run-native cosign-native cosign verify-attestation --key %s --type %s \"$1\" >/dev/null \\" % (cosign_pub, SPDX_PREDICATE_TYPE), + " _cosign verify-attestation --key %s --type %s \"$1\" >/dev/null \\" % (cosign_pub, SPDX_PREDICATE_TYPE), " && echo \"cosign: verified attestation for $1\" \\", " || { echo \"ERROR: attestation verification failed for $1\" >&2; return 1; }", ] @@ -419,7 +428,7 @@ for recipe, image in container_images.items(): for registry in registries: script += [ " for _tag in $_TAGS; do", - " oe-run-native skopeo-native skopeo copy --all%s --dest-authfile %s oci:${_OCI_MULTIARCH_OUTPUT} docker://%s/%s:${_tag}" % (digestfile, auth_config, registry, image), + " _skopeo copy --all%s --dest-authfile %s oci:${_OCI_MULTIARCH_OUTPUT} docker://%s/%s:${_tag}" % (digestfile, auth_config, registry, image), ] if cosign_key: script.append(" sign_image %s/%s:${_tag} \"$(cat \"$_DGSTFILE\")\"" % (registry, image)) From patchwork Mon Sep 21 00:41:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 98751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD5A6C982EE for ; Mon, 21 Sep 2026 00:46:02 +0000 (UTC) Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.40009.1789951562298315532 for ; Sun, 20 Sep 2026 17:46:02 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@konsulko.com header.s=google header.b=Exubl2Bp; spf=pass (domain: konsulko.com, ip: 209.85.214.178, mailfrom: tim.orling@konsulko.com) Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-2dd1dcdcf95so17228055ad.1 for ; Sun, 20 Sep 2026 17:46:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; t=1789951562; x=1790556362; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=CcEIENCTJwDG1FWVzGQzZW2aJBEQjteyoPolbkTQlpI=; b=Exubl2BpNMzjL4kvkMgNLbSCBR3MAJMBN/cRR1UKRfD3Jj6u/4CMyKakq6OAe/XNee RW+26YsggdPYTpnCz5nB8jH15SYa6w0ASQUGTjljlVnUWA+rgaPzM0u1l8xuOyzbz4U+ NeKrXRLXFo1RwKieWnwTfJbqHkQGsGdUzm3fY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789951562; x=1790556362; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=CcEIENCTJwDG1FWVzGQzZW2aJBEQjteyoPolbkTQlpI=; b=MBOz/P1L0lPQhzSst+T72lcFEB0MOTc3RbMEx+nPjZ0bMJdnrZrNzSLhY1cPcwCVNP g2qJp5JL8YIJfHqTaV0aKb/ilwGKeWGsuuVHqZxBz9LGCMoMP9yOtelKK4O1qX7LXO0e oybu6riWnIxse6avfv8BcBJNgrFZ/nG6j07un4ony/uGjMwHpHRjt604ZCwAA7pQebr3 TXqcTqndetYJwOjsLXkMbX+jqf1nJNZhZz5/LMrgjd1Cf13drlhQOSqInCRpGoYeHmeZ f2d4f5thLNLbiAHiIqSHJW8a6Y6bLfLFIwIn0d0cR4y/+KlYiebPToiys12yEbhYEnpM HZUw== X-Gm-Message-State: AFuF++k2iLeqjPPN7Vd6eAG4YIlOA0w5voFgVkZXOFgA5hmLxQWEX/Lp 8cCiIDYtRkLf1EE12Dg4LNkxs8XBdYo/A0a/uAIFmznfcMgH0bBakIE9DUvDDm2pdy+ijH4YWHr TxCMt X-Gm-Gg: AYBFou3y+RK/gas/b5W14Gp38Tev3yZ7GOGLbfDHnBbQCYl6Z44ayhKPF7J2cBcZRN9 Jv47arfYg/U60QEeaV07YZ/bOs18gxH17c3A2OxlPcgColGTnPbLwGtUzl62kbnmf4uy5lz6VBw zSvLoZ0fNl97FRlLZs4SF0AnvPSNG9+9067PY0Ohb3j8wb8Sqju50V7amfhEOEO0k7xeq+BxqGi YH4Jd5rmZIvtNeFagGmmeyURpffV0DTGe83DJM+ny5S8vw3hxBbuKxH+mlbhaGLhVXFlQBdgd3S YZ+PfDWSS5URrLxOnyrhlP+5IqQdnhknAqgDrbWXW+HGYFppCIZMwM57emsmgdzRFgD7yA++ygr woozCNhRxX83eqyT/d006sLK32pziriaLZCVebexAzMH4Nz1ard6k/Z9EovbZ2VQnSHUamT0HiM +pq+VdhUAa+rQd2QmZBb+5LNUmyCc7KMsjZcJQKQQKeMAJnGbU1lH3N0FshTHXgjNAYGaDB34n7 kn7A+W/BfXVjA== X-Received: by 2002:a17:902:868e:b0:2dd:ad7d:72e8 with SMTP id d9443c01a7336-2ddad7d74dbmr81010805ad.31.1789951561621; Sun, 20 Sep 2026 17:46:01 -0700 (PDT) Received: from thetis.home.local ([2606:c800:6024:2000:7821:4c5:938a:e12]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2ddc1803813sm24512385ad.82.2026.09.20.17.46.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Sep 2026 17:46:01 -0700 (PDT) From: tim.orling@konsulko.com To: yocto-patches@lists.yoctoproject.org Cc: Tim Orling Subject: [yocto-autobuilder-helper][PATCH 4/4] config.json: Keep the bitbake server warm for containers-library Date: Sun, 20 Sep 2026 17:41:18 -0700 Message-ID: <20260921004540.3718904-5-tim.orling@konsulko.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260921004540.3718904-1-tim.orling@konsulko.com> References: <20260921004540.3718904-1-tim.orling@konsulko.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Sep 2026 00:46:02 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/4917 From: Tim Orling The push-containers step makes ~11 bitbake invocations per container. Without BB_SERVER_TIMEOUT each starts its own cooker and reparses the whole layer stack. Set it to 600, as buildperf and metrics already do. AI-Generated: Claude Opus 5 Signed-off-by: Tim Orling --- config.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/config.json b/config.json index 22fc16d..d4b2ec2 100644 --- a/config.json +++ b/config.json @@ -1890,7 +1890,8 @@ "extravars" : [ "require conf/distro/include/meta-virt-host.conf", "DISTRO_FEATURES:append = ' usrmerge'", - "INIT_MANAGER = 'systemd'" + "INIT_MANAGER = 'systemd'", + "BB_SERVER_TIMEOUT = '600'" ], "EXTRACMDS" : ["sed -i '/meta-virt-host.conf/d' ${HELPERBUILDDIR}/conf/auto.conf"], "vcontainer" : "${VCONTAINER_TARBALL_URL}",