From patchwork Thu May 26 17:09:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52AA6C433EF for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.22513.1653585008349200418 for ; Thu, 26 May 2022 10:10:08 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DCE0E1688; Thu, 26 May 2022 10:10:07 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 706CE3F66F; Thu, 26 May 2022 10:10:07 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 01/10] python3-pluggy: add BBCLASSEXTEND for native/nativesdk Date: Thu, 26 May 2022 18:09:52 +0100 Message-Id: <20220526171001.4074388-2-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166186 This is a dependency of Hatchling, another Python build tool, so we need it natively. Signed-off-by: Ross Burton --- meta/recipes-devtools/python/python3-pluggy_1.0.0.bb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meta/recipes-devtools/python/python3-pluggy_1.0.0.bb b/meta/recipes-devtools/python/python3-pluggy_1.0.0.bb index 3f2895a061b..2935acf95fa 100644 --- a/meta/recipes-devtools/python/python3-pluggy_1.0.0.bb +++ b/meta/recipes-devtools/python/python3-pluggy_1.0.0.bb @@ -24,3 +24,5 @@ do_install_ptest() { install -d ${D}${PTEST_PATH}/testing cp -rf ${S}/testing/* ${D}${PTEST_PATH}/testing/ } + +BBCLASSEXTEND = "native nativesdk" From patchwork Thu May 26 17:09:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E35FC4167D for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web08.22671.1653585009041861381 for ; Thu, 26 May 2022 10:10:09 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 94FFA168F; Thu, 26 May 2022 10:10:08 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 31B383F66F; Thu, 26 May 2022 10:10:08 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 02/10] setuptools3: clean up class Date: Thu, 26 May 2022 18:09:53 +0100 Message-Id: <20220526171001.4074388-3-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166188 Remove a commented-out B, re-order DEPENDS< and add comments. Signed-off-by: Ross Burton --- meta/classes/setuptools3.bbclass | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/meta/classes/setuptools3.bbclass b/meta/classes/setuptools3.bbclass index 556bc801af7..66c94660d8e 100644 --- a/meta/classes/setuptools3.bbclass +++ b/meta/classes/setuptools3.bbclass @@ -1,7 +1,6 @@ inherit setuptools3-base python_pep517 -# bdist_wheel builds in ./dist -#B = "${WORKDIR}/build" +DEPENDS += "python3-setuptools-native python3-wheel-native" SETUPTOOLS_BUILD_ARGS ?= "" @@ -23,6 +22,7 @@ setuptools3_do_compile() { setuptools3_do_compile[vardepsexclude] = "MACHINE" do_compile[cleandirs] += "${PEP517_WHEEL_PATH}" +# This could be removed in the future but some recipes in meta-oe still use it setuptools3_do_install() { python_pep517_do_install } @@ -30,4 +30,3 @@ setuptools3_do_install() { EXPORT_FUNCTIONS do_configure do_compile do_install export LDSHARED="${CCLD} -shared" -DEPENDS += "python3-setuptools-native python3-wheel-native" From patchwork Thu May 26 17:09:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75718C4167B for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.22571.1653585009676254892 for ; Thu, 26 May 2022 10:10:09 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 46792150C; Thu, 26 May 2022 10:10:09 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D9C733F66F; Thu, 26 May 2022 10:10:08 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 03/10] python3-flit-core: bootstrap explicitly Date: Thu, 26 May 2022 18:09:54 +0100 Message-Id: <20220526171001.4074388-4-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166189 Add a method to python_flit_core.bbclass that does a manual build with flit explicitly, and use that to bootstrap python3-flit-core-native which can build itself. Signed-off-by: Ross Burton --- meta/classes/python_flit_core.bbclass | 5 +++++ meta/recipes-devtools/python/python3-flit-core_3.7.1.bb | 4 ++++ 2 files changed, 9 insertions(+) diff --git a/meta/classes/python_flit_core.bbclass b/meta/classes/python_flit_core.bbclass index 96652aa2043..a900f7b5579 100644 --- a/meta/classes/python_flit_core.bbclass +++ b/meta/classes/python_flit_core.bbclass @@ -3,3 +3,8 @@ inherit python_pep517 python3native python3-dir setuptools3-base DEPENDS += "python3 python3-flit-core-native" PEP517_BUILD_API = "flit_core.buildapi" + +python_flit_core_do_manual_build () { + cd ${PEP517_SOURCE_PATH} + nativepython3 -m flit_core.wheel --outdir ${PEP517_WHEEL_PATH} . +} diff --git a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb index 8d107384a6d..abe620374c0 100644 --- a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb +++ b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb @@ -20,6 +20,10 @@ DEPENDS:append:class-native = " unzip-native" PYPI_PACKAGE = "flit" PEP517_SOURCE_PATH = "${S}/flit_core" +do_compile:class-native () { + python_flit_core_do_manual_build +} + do_install:class-native () { python_pep517_do_bootstrap_install } From patchwork Thu May 26 17:09:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 740C1C43219 for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.22571.1653585009676254892 for ; Thu, 26 May 2022 10:10:10 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E98A61688; Thu, 26 May 2022 10:10:09 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 896323F66F; Thu, 26 May 2022 10:10:09 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 04/10] python3-installer: bootstrap by installing installer with installer Date: Thu, 26 May 2022 18:09:55 +0100 Message-Id: <20220526171001.4074388-5-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166190 The installer can install itself by simply adding src/ to PYTHONPATH. As this previously used unzip, we need to disable bytecode as otherwise the newly packaged .pyc files conflict with the any generated .pyc files in the sysroot. Signed-off-by: Ross Burton --- meta/recipes-devtools/python/python3-installer_0.5.1.bb | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/meta/recipes-devtools/python/python3-installer_0.5.1.bb b/meta/recipes-devtools/python/python3-installer_0.5.1.bb index f4f9e1bde64..b7780702d6b 100644 --- a/meta/recipes-devtools/python/python3-installer_0.5.1.bb +++ b/meta/recipes-devtools/python/python3-installer_0.5.1.bb @@ -12,11 +12,13 @@ SRC_URI[sha256sum] = "f970995ec2bb815e2fdaf7977b26b2091e1e386f0f42eafd5ac811953d inherit pypi python_flit_core +# Bootstrap the native install by installing with ourself DEPENDS:remove:class-native = "python3-installer-native" -DEPENDS:append:class-native = " unzip-native" -do_install:class-native () { - python_pep517_do_bootstrap_install +INSTALL_WHEEL_COMPILE_BYTECODE:class-native = "--no-compile-bytecode" + +do_install:prepend:class-native() { + export PYTHONPATH="${S}/src" } BBCLASSEXTEND = "native nativesdk" From patchwork Thu May 26 17:09:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60B76C433FE for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.22572.1653585011391890402 for ; Thu, 26 May 2022 10:10:11 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 96044150C; Thu, 26 May 2022 10:10:10 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3691A3F66F; Thu, 26 May 2022 10:10:10 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 05/10] python3-pep517: add new recipe Date: Thu, 26 May 2022 18:09:56 +0100 Message-Id: <20220526171001.4074388-6-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166191 This is a runtime dependency of most Python package build frontends. Signed-off-by: Ross Burton --- meta/recipes-devtools/python/python3-pep517_0.12.0.bb | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-pep517_0.12.0.bb diff --git a/meta/recipes-devtools/python/python3-pep517_0.12.0.bb b/meta/recipes-devtools/python/python3-pep517_0.12.0.bb new file mode 100644 index 00000000000..43ac35a3f79 --- /dev/null +++ b/meta/recipes-devtools/python/python3-pep517_0.12.0.bb @@ -0,0 +1,10 @@ +SUMMARY = "Wrappers to build Python packages using PEP 517 hooks" +HOMEPAGE = "https://github.com/pypa/pep517" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=aad69c93f605003e3342b174d9b0708c" + +SRC_URI[sha256sum] = "931378d93d11b298cf511dd634cf5ea4cb249a28ef84160b3247ee9afb4e8ab0" + +inherit pypi python_flit_core + +BBCLASSEXTEND = "native nativesdk" \ No newline at end of file From patchwork Thu May 26 17:09:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62D66C43217 for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.22571.1653585009676254892 for ; Thu, 26 May 2022 10:10:11 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 45BFC1688; Thu, 26 May 2022 10:10:11 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DA57A3F66F; Thu, 26 May 2022 10:10:10 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 06/10] python3-setuptools-scm: DEPEND on python3-packaging Date: Thu, 26 May 2022 18:09:57 +0100 Message-Id: <20220526171001.4074388-7-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166192 This is a build dependency that isn't flagged when we call the build API directly. Signed-off-by: Ross Burton --- meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb b/meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb index 9aaae071d31..212aa8a005a 100644 --- a/meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb +++ b/meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb @@ -11,7 +11,7 @@ inherit pypi python_setuptools_build_meta UPSTREAM_CHECK_REGEX = "setuptools_scm-(?P.*)\.tar" -DEPENDS += "python3-tomli-native" +DEPENDS += "python3-tomli-native python3-packaging-native" RDEPENDS:${PN} = "\ ${PYTHON_PN}-packaging \ From patchwork Thu May 26 17:09:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8536 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6657CC4321E for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.22514.1653585012276439434 for ; Thu, 26 May 2022 10:10:12 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E8C4B150C; Thu, 26 May 2022 10:10:11 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 88F383F66F; Thu, 26 May 2022 10:10:11 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 07/10] python3-picobuild: add new recipe Date: Thu, 26 May 2022 18:09:58 +0100 Message-Id: <20220526171001.4074388-8-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166193 Picobuild is a pico-scale Python PEP517 build frontend, designed to have minimal dependencies (via vendoring) to be well suited for building source-based distributions, such as OpenEmbedded. Signed-off-by: Ross Burton --- .../python/python3-picobuild_0.1.bb | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-picobuild_0.1.bb diff --git a/meta/recipes-devtools/python/python3-picobuild_0.1.bb b/meta/recipes-devtools/python/python3-picobuild_0.1.bb new file mode 100644 index 00000000000..bbd3636eea8 --- /dev/null +++ b/meta/recipes-devtools/python/python3-picobuild_0.1.bb @@ -0,0 +1,25 @@ +SUMMARY = "Library and tool for installing Python wheels" + +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=6c811a9fbdf5641ff0b0d43fbbb440e5" +# TODO vendorered license files + +SRC_URI = "git://gitlab.com/rossburton/picobuild.git;protocol=https;branch=main" +SRCREV = "f08fe0375e2b93e3363416f210534d1ac3f8fc68" + +S = "${WORKDIR}/git" + +inherit python_flit_core + +# For target builds we can deleted the vendored libraries and RDEPEND on them +# instead. Use an explicit list to be sure we delete what we RDEPEND on. +do_install:append:class-target() { + rm -rf ${D}${PYTHON_SITEPACKAGES_DIR}/picobuild/vendored/packaging + rm -rf ${D}${PYTHON_SITEPACKAGES_DIR}/picobuild/vendored/pep517* + rm -rf ${D}${PYTHON_SITEPACKAGES_DIR}/picobuild/vendored/pyparsing + rm -rf ${D}${PYTHON_SITEPACKAGES_DIR}/picobuild/vendored/tomli +} + +RDEPENDS:${PN}:append:class-target = " python3-tomli python3-pep517 python3-packaging python3-pyparsing" + +BBCLASSEXTEND = "native nativesdk" From patchwork Thu May 26 17:09:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8538 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50ADAC433F5 for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.22573.1653585012942573684 for ; Thu, 26 May 2022 10:10:13 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 98BB81688; Thu, 26 May 2022 10:10:12 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 397B73F66F; Thu, 26 May 2022 10:10:12 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 08/10] python_pep517: use picobuild instead of manually calling the API Date: Thu, 26 May 2022 18:09:59 +0100 Message-Id: <20220526171001.4074388-9-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166194 Calling the PEP-517 API directly mostly works, but sometimes doesn't. For example we don't verify build dependencies, which led to the cbor2 ugprade silently failing to actually package anything. The standard frontend is pypa/build, but for source-based distributions that can be annoying to build as it depends on the following packages: - tomli - pep517 - packaging - pyparsing Manually bootstrapping those recipes is possible, but tedious. Picobuild is another frontend (written by myself) which is designed explicitly to be used programatically by source-based distributions: it doesn't support builds inside virtual environments as we're building distribution packages, and it vendors the dependencies for bootstrapping if they're not available. Over time more packages are expected to move to using Flit to build which makes the bootstrapping process slightly easier, and tomli will be integrated into Python 3.11, so it's possible that in the future we drop picobuild and switch to build. This change means the PEP517_BUILD_API variable is obsolete, so remove it. Signed-off-by: Ross Burton --- meta/classes/python_pep517.bbclass | 8 ++------ meta/recipes-devtools/python/python3-flit-core_3.7.1.bb | 2 +- meta/recipes-devtools/python/python3-installer_0.5.1.bb | 8 ++++++-- meta/recipes-devtools/python/python3-picobuild_0.1.bb | 2 ++ 4 files changed, 11 insertions(+), 9 deletions(-) diff --git a/meta/classes/python_pep517.bbclass b/meta/classes/python_pep517.bbclass index 34ffdc9c0db..9c37db6be60 100644 --- a/meta/classes/python_pep517.bbclass +++ b/meta/classes/python_pep517.bbclass @@ -4,14 +4,11 @@ # This class will build a wheel in do_compile, and use pypa/installer to install # it in do_install. -DEPENDS:append = " python3-installer-native" +DEPENDS:append = " python3-picobuild-native python3-installer-native" # Where to execute the build process from PEP517_SOURCE_PATH ?= "${S}" -# The PEP517 build API entry point -PEP517_BUILD_API ?= "unset" - # The directory where wheels will be written PEP517_WHEEL_PATH ?= "${WORKDIR}/dist" @@ -31,8 +28,7 @@ python_pep517_do_configure () { # When we have Python 3.11 we can parse pyproject.toml to determine the build # API entry point directly python_pep517_do_compile () { - cd ${PEP517_SOURCE_PATH} - nativepython3 -c "import ${PEP517_BUILD_API} as api; api.build_wheel('${PEP517_WHEEL_PATH}')" + nativepython3 -m picobuild --wheel --source ${PEP517_SOURCE_PATH} --dest ${PEP517_WHEEL_PATH} } do_compile[cleandirs] += "${PEP517_WHEEL_PATH}" diff --git a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb index abe620374c0..fe6b16ac540 100644 --- a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb +++ b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb @@ -13,7 +13,7 @@ SRC_URI[sha256sum] = "3c9bd9c140515bfe62dd938c6610d10d6efb9e35cc647fc614fe5fb3a5 inherit pypi python_flit_core # Need to install by hand as there's a dependency loop -DEPENDS:remove:class-native = " python3-installer-native" +DEPENDS:remove:class-native = " python3-picobuild-native python3-installer-native" DEPENDS:append:class-native = " unzip-native" # We need the full flit tarball diff --git a/meta/recipes-devtools/python/python3-installer_0.5.1.bb b/meta/recipes-devtools/python/python3-installer_0.5.1.bb index b7780702d6b..07bbafa81fb 100644 --- a/meta/recipes-devtools/python/python3-installer_0.5.1.bb +++ b/meta/recipes-devtools/python/python3-installer_0.5.1.bb @@ -12,11 +12,15 @@ SRC_URI[sha256sum] = "f970995ec2bb815e2fdaf7977b26b2091e1e386f0f42eafd5ac811953d inherit pypi python_flit_core -# Bootstrap the native install by installing with ourself -DEPENDS:remove:class-native = "python3-installer-native" +# Bootstrap the native build +DEPENDS:remove:class-native = "python3-picobuild-native python3-installer-native" INSTALL_WHEEL_COMPILE_BYTECODE:class-native = "--no-compile-bytecode" +do_compile:class-native () { + python_flit_core_do_manual_build +} + do_install:prepend:class-native() { export PYTHONPATH="${S}/src" } diff --git a/meta/recipes-devtools/python/python3-picobuild_0.1.bb b/meta/recipes-devtools/python/python3-picobuild_0.1.bb index bbd3636eea8..49aebf81032 100644 --- a/meta/recipes-devtools/python/python3-picobuild_0.1.bb +++ b/meta/recipes-devtools/python/python3-picobuild_0.1.bb @@ -11,6 +11,8 @@ S = "${WORKDIR}/git" inherit python_flit_core +DEPENDS:remove:class-native = "python3-picobuild-native" + # For target builds we can deleted the vendored libraries and RDEPEND on them # instead. Use an explicit list to be sure we delete what we RDEPEND on. do_install:append:class-target() { From patchwork Thu May 26 17:10:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 544D1C4332F for ; Thu, 26 May 2022 17:10:14 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web10.22472.1653585013606650275 for ; Thu, 26 May 2022 10:10:13 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 475D0150C; Thu, 26 May 2022 10:10:13 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DCF083F66F; Thu, 26 May 2022 10:10:12 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 09/10] classes: remove obsolete PEP517_BUILD_API Date: Thu, 26 May 2022 18:10:00 +0100 Message-Id: <20220526171001.4074388-10-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166195 This variable is no longer used, so remove it from the python_* classes. Signed-off-by: Ross Burton --- meta/classes/python_flit_core.bbclass | 2 -- meta/classes/python_poetry_core.bbclass | 2 -- meta/classes/python_setuptools_build_meta.bbclass | 2 -- 3 files changed, 6 deletions(-) diff --git a/meta/classes/python_flit_core.bbclass b/meta/classes/python_flit_core.bbclass index a900f7b5579..7109307de5b 100644 --- a/meta/classes/python_flit_core.bbclass +++ b/meta/classes/python_flit_core.bbclass @@ -2,8 +2,6 @@ inherit python_pep517 python3native python3-dir setuptools3-base DEPENDS += "python3 python3-flit-core-native" -PEP517_BUILD_API = "flit_core.buildapi" - python_flit_core_do_manual_build () { cd ${PEP517_SOURCE_PATH} nativepython3 -m flit_core.wheel --outdir ${PEP517_WHEEL_PATH} . diff --git a/meta/classes/python_poetry_core.bbclass b/meta/classes/python_poetry_core.bbclass index 577663b8f17..0aaf66b1944 100644 --- a/meta/classes/python_poetry_core.bbclass +++ b/meta/classes/python_poetry_core.bbclass @@ -1,5 +1,3 @@ inherit python_pep517 python3native setuptools3-base DEPENDS += "python3-poetry-core-native" - -PEP517_BUILD_API = "poetry.core.masonry.api" diff --git a/meta/classes/python_setuptools_build_meta.bbclass b/meta/classes/python_setuptools_build_meta.bbclass index b2bba35a0bc..974054fe5ac 100644 --- a/meta/classes/python_setuptools_build_meta.bbclass +++ b/meta/classes/python_setuptools_build_meta.bbclass @@ -1,5 +1,3 @@ inherit setuptools3-base python_pep517 DEPENDS += "python3-setuptools-native python3-wheel-native" - -PEP517_BUILD_API = "setuptools.build_meta" From patchwork Thu May 26 17:10:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 8541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BEDBC433EF for ; Thu, 26 May 2022 17:10:24 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.22516.1653585014406904572 for ; Thu, 26 May 2022 10:10:14 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E8C01150C; Thu, 26 May 2022 10:10:13 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8A0FA3F66F; Thu, 26 May 2022 10:10:13 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [RFC PATCH 10/10] documentation: remove obsolete PEP517_BUILD_API Date: Thu, 26 May 2022 18:10:01 +0100 Message-Id: <20220526171001.4074388-11-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220526171001.4074388-1-ross.burton@arm.com> References: <20220526171001.4074388-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 26 May 2022 17:10:24 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166196 This variable has been obsoleted, so remove it from the documentation. Signed-off-by: Ross Burton --- documentation/ref-manual/classes.rst | 4 +--- documentation/ref-manual/variables.rst | 5 ----- 2 files changed, 1 insertion(+), 8 deletions(-) diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index 729aa259e02..e170d31cd9d 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst @@ -2000,9 +2000,7 @@ The ``python_pep517`` class builds and installs a Python ``wheel`` binary archive (see `PEP-517 `__). Recipes wouldn't inherit this directly, instead typically another class will -inherit this, add the relevant native dependencies, and set -:term:`PEP517_BUILD_API` to the Python class which implements the PEP-517 build -API. +inherit this and add the relevant native dependencies. Examples of classes which do this are :ref:`python_flit_core `, :ref:`python_setuptools_build_meta diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index 367b4674e2c..da9b8577842 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst @@ -5657,11 +5657,6 @@ system and gives an overview of their function and contents. :term:`PE` is the default value of the :term:`PKGE` variable. - :term:`PEP517_BUILD_API` - When used by recipes that inherit the :ref:`python_pep517 - ` class, specifies the entry point to the - PEP-517 compliant build API (such as ``flit_core.buildapi``). - :term:`PEP517_WHEEL_PATH` When used by recipes that inherit the :ref:`python_pep517 ` class,