From patchwork Tue Mar 31 14:53:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Woolley X-Patchwork-Id: 84918 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AB6C109B48B for ; Tue, 31 Mar 2026 14:53:33 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.22396.1774968803439175738 for ; Tue, 31 Mar 2026 07:53:23 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@windriver.com header.s=PPS06212021 header.b=O8k4nkbp; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=85502649bd=rob.woolley@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62V4w3D6274792; Tue, 31 Mar 2026 14:53:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=cc:content-transfer-encoding:content-type:date:from :in-reply-to:message-id:mime-version:references:subject:to; s= PPS06212021; bh=dYd9AtrqXVzG3kkPmBJJveSdb9jhz/Z9cXLbO2WtMGs=; b= O8k4nkbpn/FhxaaBjmaWYfiMRFGg9KZOEZp3gVfkEzIv+A3uDqtl3e0YUg4plS3b kYaHn8eV6H2Nt6K29HI/mJfZMqac2/SCIKQv6FEnnKohWixkiznHdUGYJV2VivXC Yq7JYJofbVXT0mQIWplvXgN9WCpcsOznECPw4l2OwqebdQFqAPzVmNdIcBi7Zqbj Mbu7f63vtwoTxU+Z234bbPSJNiEVv/eXMsZmX7bvmetYCAEHh2gjzgntSyviWT3L KO1uNFMx3Zf809uYdi5ioHviz9M3LbcJvpEJqBR4wVE0EWIdCSCfklmVwtud/G7n GVM/cWxdcp9xqX7/tkdJ4w== Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [128.224.246.36]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4d65y4btqq-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 31 Mar 2026 14:53:21 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (10.11.224.121) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.61; Tue, 31 Mar 2026 07:53:19 -0700 Received: from ala-lpggp3.wrs.com (10.11.232.110) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server id 15.1.2507.61 via Frontend Transport; Tue, 31 Mar 2026 07:53:19 -0700 From: Rob Woolley To: CC: Subject: [PATCH 1/6] bitbake: Add checks for importing bb module Date: Tue, 31 Mar 2026 07:53:14 -0700 Message-ID: <20260331145319.3125456-2-rob.woolley@windriver.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20260331145319.3125456-1-rob.woolley@windriver.com> References: <20260331145319.3125456-1-rob.woolley@windriver.com> MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=QaZrf8bv c=1 sm=1 tr=0 ts=69cbdfe1 cx=c_pps a=AbJuCvi4Y3V6hpbCNWx0WA==:117 a=AbJuCvi4Y3V6hpbCNWx0WA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=bi6dqmuHe4P4UrxVR6um:22 a=fTW__CHxibyLmBMfj2wP:22 a=t7CeM3EgAAAA:8 a=vUYKkVZVldypNouynCYA:9 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzMxMDE0NCBTYWx0ZWRfXxcND7J1RD301 BMCAm7NGnaGsXBvZhPKQ8XRDOmUgBhvGscoG3GI/VecgRVy2JTRzFjhVOVx6vi7mtyQaX72D043 220mrd0SMlLXGC2qi1RkeRicvV3AaMLSSgWRZlIBRtMVvOVN3HLncyimCy4yrK+pn469I0QTWyE p1j0BishUOP/Mk1SasH3l5ssKFjgqRN16+MuNt9fh9THnXWLjCWSEhFuCkmWLvroGIYLU++GJuO ycfBjIfyvf8n5UGURBkTBZzoFApEf4ksVlnQX5PZtyDbjwwVRt1V/Fhr3IHLMDtjSgPKy6VdqVU WNPeaDJNxDVHS9ixc2B4asD232owEd2/Yz0W+Fsp6AoE9diHj5Jroi5rNGKPt6c5SEiKIlePliS Sccxlzo95gpJGtdZuXoPpHMiixU6gahCVTd+4CxLYzLmIlRi0dnAFG2FVDZiEsezpJ790gGtgH0 KEYDxltk6X4V84tTzLA== X-Proofpoint-GUID: Chp881F-t2iAqtSQyyqEZxuqzCmOuhYf X-Proofpoint-ORIG-GUID: Chp881F-t2iAqtSQyyqEZxuqzCmOuhYf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-31_03,2026-03-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 adultscore=0 suspectscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603310144 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 31 Mar 2026 14:53:33 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19293 The __version__ has been present since bbmake to ensure that the Python script uses the appropriate bb module. This was necessary when bbmake had functional code that had to match the bb module. The code has since been moved into bb/main.py. The skeleton code left in bin/bitbake rarely changes save for bumping the version. When executed in place, the bitbake script manipulates sys.path to find the bb module in ../lib. This path is inserted to the beginning of sys.path and it cannot be overridden. This path is inserted regardless of whether the bb module is present in the lib directory. This adds a check for the existence of the lib/bb/__init__.py file before inserting the directory to the beginning of sys.path. This supports bitbake to be executed from inside the git repository. It deliberately fails in the situation where lib/bb/__init__.py is missing, but the bb package exists inside a site-package directory. This situation was previously possible. Signed-off-by: Rob Woolley --- bin/bitbake | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/bin/bitbake b/bin/bitbake index a995bd66..76ec80d9 100755 --- a/bin/bitbake +++ b/bin/bitbake @@ -15,23 +15,27 @@ import sys import warnings warnings.simplefilter("default") -sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)), - 'lib')) -try: - import bb -except RuntimeError as exc: - sys.exit(str(exc)) +bindir = os.path.abspath(os.path.dirname(__file__)) +libdir = os.path.join(os.path.dirname(bindir), 'lib') +bbfile = os.path.join(libdir, 'bb', '__init__.py') + +if os.path.exists(bbfile): + # Execute bitbake in git repository + try: + sys.path[0:0] = [libdir] + import bb + except ImportError as e: + print(f"Could not import bb from lib: {e}", file=sys.stderr) +else: + print(f"Could not find bb module", file=sys.stderr) + sys.exit(1) from bb import cookerdata from bb.main import bitbake_main, BitBakeConfigParameters, BBMainException bb.utils.check_system_locale() -__version__ = "2.16.0" - if __name__ == "__main__": - if __version__ != bb.__version__: - sys.exit("Bitbake core version and program version mismatch!") try: sys.exit(bitbake_main(BitBakeConfigParameters(sys.argv), cookerdata.CookerConfiguration())) From patchwork Tue Mar 31 14:53:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Woolley X-Patchwork-Id: 84920 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48388109B48D for ; Tue, 31 Mar 2026 14:53:33 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.22399.1774968804570671735 for ; Tue, 31 Mar 2026 07:53:24 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@windriver.com header.s=PPS06212021 header.b=UY0uo9+M; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=85502649bd=rob.woolley@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62V4w3D7274792; Tue, 31 Mar 2026 14:53:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=cc:content-transfer-encoding:content-type:date:from :in-reply-to:message-id:mime-version:references:subject:to; s= PPS06212021; bh=H7PQl330GoWBeL42ELu2uLcYC20QPtFdNQs/YFELTHk=; b= UY0uo9+MXTXahwj6v/HYl7hf/xBwLhbyYPwxROKIHW8qgdXyp2tG+1GAIRfP4mZs oyLc66RBCVaGiNZ9wooJU9gQ6/xNS+3pfJkvbsbrvsWyjdEmbRFgnrpFjUe3nY/3 XOUxMjK9NBxuz1GVyUWQQGf6C/JCggX/jae1HsAWnrrRz2geLiAhfF8LkMS++bL4 7ohrxS/mA3X0bmL79J/Ci3mDU9FbSqRWH8prUphmr2g8E7GYoyB15QKEjz1c53QY qIT27dWw0aw9c6C3+2hdZ6qmWs/PDJBSTlNIEF14cqjqbiSgYpQKX2281a3hZwn3 8WWinu73EKt8DG1goNKuuw== Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [128.224.246.36]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4d65y4btqq-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 31 Mar 2026 14:53:23 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (10.11.224.121) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.61; Tue, 31 Mar 2026 07:53:19 -0700 Received: from ala-lpggp3.wrs.com (10.11.232.110) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server id 15.1.2507.61 via Frontend Transport; Tue, 31 Mar 2026 07:53:19 -0700 From: Rob Woolley To: CC: Subject: [PATCH 2/6] pypi: Add PyPI packaging for bitbake-setup Date: Tue, 31 Mar 2026 07:53:15 -0700 Message-ID: <20260331145319.3125456-3-rob.woolley@windriver.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20260331145319.3125456-1-rob.woolley@windriver.com> References: <20260331145319.3125456-1-rob.woolley@windriver.com> MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=QaZrf8bv c=1 sm=1 tr=0 ts=69cbdfe3 cx=c_pps a=AbJuCvi4Y3V6hpbCNWx0WA==:117 a=AbJuCvi4Y3V6hpbCNWx0WA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=bi6dqmuHe4P4UrxVR6um:22 a=fTW__CHxibyLmBMfj2wP:22 a=24AZYWMyAAAA:8 a=iGHA9ds3AAAA:8 a=Q4-j1AaZAAAA:8 a=t7CeM3EgAAAA:8 a=qV09NasGAAAA:8 a=V5MqrhjLj9XmrIPkEmEA:9 a=bG88sKzkDEFeXWNnvthB:22 a=nM-MV4yxpKKO9kiQg6Ot:22 a=9H3Qd4_ONW2Ztcrla5EB:22 a=FdTzh2GWekK77mhwV6Dw:22 a=GlicbclHOgpI_Rq0ze_Y:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzMxMDE0NCBTYWx0ZWRfX2Wf530dqifCR DTYufFFXam+uTZzuaGR8XHNd9YnzroSVzeusthax1WP+pp6jH9L9m8+JdF2Y8nwqnrkq1Du39D+ 0MaPM8or9rpyFNpv5GGIJpwV36Y4F/oY1mH1W2zVZlWBE4lolKnV5+f+hZzH/kd1puRJRbkPWRi bu1LFwfNzsgT+uat+kRjMu7l1Obqwfu4ek4Wc2Q2++s+p3pfTZPGPzafZ/D8Z5UUpXYLXVHvR8M /sXaqCtpPvRGgxV2+CFQjcZ2x3g5e6/txdK/qSLkXK/1vti1UmrbZb18M0d+lUx/j1R1rJ7j6fZ Q7TUki9/4b3n3xWqhjLOryP5utNQCvESXMz2n/KEppZSdVIrhsHHMCIIqD197tf6p3vOnDk6PBQ hEmDm7mijxfJE1Q4gSDzxK46mr/xPTTh1zusa/fHx/SC4TpBCnwW73leUc2HGtY9qbXFwaJCHWy v0k8sFO4UrBG9ZsLNdQ== X-Proofpoint-GUID: r7KhxK0gzphQVJta-ek6hRuUfdUfwJ9V X-Proofpoint-ORIG-GUID: r7KhxK0gzphQVJta-ek6hRuUfdUfwJ9V X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-31_03,2026-03-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 adultscore=0 suspectscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603310144 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 31 Mar 2026 14:53:33 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19294 We wish to publish bitbake-setup to PyPI to improve the workflow for new users that are familiar with pip install. In order to publish bitbake-setup as a standalone package, we must create a staging directory for Python to build the package. The package-bitbake-setup.py automates the staging of the necessary files. You may supply your desired directory as the first argument, but packaging_workspace in the root of the git repository is chosen by default. The packaging process and related tests have also been included as part of bitbake-selftest and lib/bb/tests/setup.py. These tests use package-bitbake-setup.py. The tests include: * Verify bitbake-setup --help runs successfully. * Verify bitbake-setup list runs successfully. * Verify console script entry points are correctly defined. * Verify all expected modules can be imported from installed package. * Verify package metadata is correctly set. * Verify vendored dependencies (bs4, ply, progressbar, simplediff) are not bundled in package. * Verify version is set correctly (not fallback 0.0.0 unless expected). * Verify wheel METADATA file contains required fields. The pyproject.toml, LICENSE, and README.md files are used to create the package and provide information for PyPI. Assisted-by: Claude:claude-4.6-opus Signed-off-by: Rob Woolley --- bin/bitbake-selftest | 2 + contrib/pypi/LICENSE | 9 + contrib/pypi/README.md | 42 ++++ contrib/pypi/package-bitbake-setup.py | 79 +++++++ contrib/pypi/pyproject.toml | 92 ++++++++ lib/bb/tests/setup.py | 289 +++++++++++++++++++++++++- 6 files changed, 512 insertions(+), 1 deletion(-) create mode 100644 contrib/pypi/LICENSE create mode 100644 contrib/pypi/README.md create mode 100755 contrib/pypi/package-bitbake-setup.py create mode 100644 contrib/pypi/pyproject.toml diff --git a/bin/bitbake-selftest b/bin/bitbake-selftest index fb7c57dd..7b5d68ff 100755 --- a/bin/bitbake-selftest +++ b/bin/bitbake-selftest @@ -67,6 +67,8 @@ ENV_HELP = """\ Environment variables: BB_SKIP_NETTESTS set to 'yes' in order to skip tests using network connection + BB_SKIP_PYPI_TESTS set to 'no' to run PyPI packaging tests + (default: yes/skip) BB_TMPDIR_NOCLEAN set to 'yes' to preserve test tmp directories """ diff --git a/contrib/pypi/LICENSE b/contrib/pypi/LICENSE new file mode 100644 index 00000000..f9b44182 --- /dev/null +++ b/contrib/pypi/LICENSE @@ -0,0 +1,9 @@ +bitbake-setup is licensed under the GNU General Public License version 2.0. See +LICENSE.GPL-2.0-only for further details. + +Individual files contain the following style tags instead of the full license text: + + SPDX-License-Identifier: GPL-2.0-only + +This enables machine processing of license information based on the SPDX +License Identifiers that are here available: http://spdx.org/licenses/ diff --git a/contrib/pypi/README.md b/contrib/pypi/README.md new file mode 100644 index 00000000..a6dd99e0 --- /dev/null +++ b/contrib/pypi/README.md @@ -0,0 +1,42 @@ +# bitbake-setup + +This package provides the `bitbake-setup` command and the Python modules +required to support BitBake setup and configuration. + +## Usage + +Instructions on uses of bitbake-setup can be found in +[Setting Up the Environment With bitbake-setup](https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/bitbake-user-manual-environment-setup.html) from the Yocto Project manual. + +List the available configurations; +```bash +bitbake-setup list +``` + +Show the help for the bitbake-setup init subcommand: +```bash +bitbake-setup init --help +usage: bitbake-setup init [-h] [--non-interactive] [--source-overrides SOURCE_OVERRIDES] [--setup-dir-name SETUP_DIR_NAME] [--skip-selection SKIP_SELECTION] [-L SOURCE_NAME PATH] + [config ...] + +positional arguments: + config path/URL/id to a configuration file (use 'list' command to get available ids), followed by configuration options. Bitbake-setup will ask to choose from available + choices if command line doesn't completely specify them. + +options: + -h, --help show this help message and exit + --non-interactive Do not ask to interactively choose from available options; if bitbake-setup cannot make a decision it will stop with a failure. + --source-overrides SOURCE_OVERRIDES + Override sources information (repositories/revisions) with values from a local json file. + --setup-dir-name SETUP_DIR_NAME + A custom setup directory name under the top directory. + --skip-selection SKIP_SELECTION + Do not select and set an option/fragment from available choices; the resulting bitbake configuration may be incomplete. + -L SOURCE_NAME PATH, --use-local-source SOURCE_NAME PATH + Symlink local source into a build, instead of getting it as prescribed by a configuration (useful for local development). +``` + +To initialize a workspace for the Poky reference distro using the development branch (ie. "master") of OpenEmbedded: +```bash +bitbake-setup init poky-master +``` \ No newline at end of file diff --git a/contrib/pypi/package-bitbake-setup.py b/contrib/pypi/package-bitbake-setup.py new file mode 100755 index 00000000..60c503c4 --- /dev/null +++ b/contrib/pypi/package-bitbake-setup.py @@ -0,0 +1,79 @@ +#!/usr/bin/env python3 + +import argparse +import logging +import shutil +from pathlib import Path + + +def create_packaging_workspace(directory): + # Create the directory for packaging workspace + if len(directory or "") > 0: + workspace_dir = Path(directory) + else: + # This script is located in contrib/pypi/package-bitbake-setup.py + workspace_dir = Path(__file__).parents[2] / "packaging_workspace" + + if not workspace_dir.exists(): + logging.debug(f"Created packaging workspace at: {workspace_dir}") + workspace_dir.mkdir(exist_ok=True) + else: + logging.debug(f"Packaging workspace already exists at: {workspace_dir}") + + # Copy packaging files to the workspace + files_to_copy = [ + "contrib/pypi/pyproject.toml", + "contrib/pypi/README.md", + "contrib/pypi/LICENSE", + "LICENSE.MIT", + "LICENSE.GPL-2.0-only" + ] + + for file in files_to_copy: + src_path = Path(__file__).parents[2] / file + dest_path = workspace_dir / Path(file).name + + if src_path.is_dir(): + shutil.copytree(src_path, dest_path, dirs_exist_ok=True) + logging.debug(f"Copied directory: {src_path} to {dest_path}") + else: + shutil.copy2(src_path, dest_path) + logging.debug(f"Copied file: {src_path} to {dest_path}") + + + # Copy necessary modules to the workspace + modules_to_bundle = [ + "lib/bb", + ] + + for module in modules_to_bundle: + src_path = Path(__file__).parents[2] / module + dest_path = workspace_dir / "src" / Path(module).name + + if src_path.is_dir(): + shutil.copytree(src_path, dest_path, dirs_exist_ok=True) + logging.debug(f"Bundled module directory: {src_path} to {dest_path}") + else: + shutil.copy2(src_path, dest_path) + logging.debug(f"Bundled module file: {src_path} to {dest_path}") + + # Create bitbake_setup module + bitbake_setup_dir = Path(workspace_dir / "src" / "bitbake_setup") + bitbake_setup_dir.mkdir(exist_ok=True) + Path(bitbake_setup_dir / "__init__.py").touch() + shutil.copy2(Path(__file__).parents[2] / "bin" / "bitbake-setup", str(bitbake_setup_dir / "__main__.py")) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') + + parser = argparse.ArgumentParser(description='Package bitbake-setup for PyPI') + parser.add_argument('-v', '--verbose', action='store_true', help='increase output verbosity.') + parser.add_argument('-d', '--directory', type=str, help='specify the directory to create the packaging workspace in.') + + args = parser.parse_args() + + if args.verbose: + logging.getLogger().setLevel(logging.DEBUG) + + create_packaging_workspace(args.directory) diff --git a/contrib/pypi/pyproject.toml b/contrib/pypi/pyproject.toml new file mode 100644 index 00000000..69f34667 --- /dev/null +++ b/contrib/pypi/pyproject.toml @@ -0,0 +1,92 @@ +[build-system] +requires = [ + "setuptools>=64", +] +build-backend = "setuptools.build_meta" + +[project] +name = "bitbake-setup" +dynamic = ["version"] +description = "bitbake-setup" +readme = "README.md" +requires-python = ">=3.9" +license = "GPL-2.0-only AND MIT" +authors = [ + { name = "OpenEmbedded BitBake Developers", email = "bitbake-devel@lists.openembedded.org" } +] +classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: POSIX :: Linux", +] + +dependencies = [ + # bitbake-setup is mostly self-contained +] + +[project.optional-dependencies] +test = [ + "pytest>=7", +] + +lint = [ + "ruff>=0.3", + "mypy>=1.8", +] + +dev = [ + "pytest>=7", + "ruff>=0.3", + "mypy>=1.8", + "build", + "twine", +] + +[project.scripts] +bitbake-setup = "bitbake_setup.__main__:main" + +[project.urls] +Homepage = "https://git.openembedded.org/bitbake/" +Documentation = "https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-environment-setup.html" +Repository = "https://git.openembedded.org/bitbake/" + +[tool.mypy] +python_version = "3.9" +warn_unused_configs = true +ignore_missing_imports = true +no_implicit_optional = true +check_untyped_defs = false + +[tool.ruff] +target-version = "py39" +line-length = 88 +src = ["src"] + +[tool.ruff.lint] +select = [ + "E", # pycodestyle + "F", # pyflakes + "B", # flake8-bugbear + "I", # import sorting +] +ignore = [ + "E501", # line length (handled by formatter) +] + +[tool.ruff.format] +quote-style = "double" +indent-style = "space" + +[tool.setuptools] +package-dir = { "" = "src" } +zip-safe = false +include-package-data = true + +[tool.setuptools.dynamic] +version = {attr = "bb.__version__"} + +[tool.setuptools.packages.find] +where = ["src"] +include = [ + "bb*", + "bitbake_setup" +] diff --git a/lib/bb/tests/setup.py b/lib/bb/tests/setup.py index 638d56d3..b031270e 100644 --- a/lib/bb/tests/setup.py +++ b/lib/bb/tests/setup.py @@ -11,7 +11,14 @@ import glob import hashlib import json import os +import shutil import stat +import subprocess +import sys +import tempfile +import unittest +import venv +import zipfile from bb.tests.support.httpserver import HTTPService class BitbakeSetupTest(FetcherTest): @@ -92,7 +99,11 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"])) def runbbsetup(self, cmd): bbsetup = os.path.abspath(os.path.dirname(__file__) + "/../../../bin/bitbake-setup") - return bb.process.run("{} --global-settings {} {}".format(bbsetup, os.path.join(self.tempdir, 'global-config'), cmd)) + # Set PYTHONPATH so subprocess can find bb module instead of relying on the current directory + env = os.environ.copy() + libdir = os.path.abspath(os.path.dirname(__file__) + "/../..") + env["PYTHONPATH"] = libdir + ":" + env.get("PYTHONPATH", "") + return bb.process.run("{} --global-settings {} {}".format(bbsetup, os.path.join(self.tempdir, 'global-config'), cmd), env=env, cwd=self.tempdir) def _add_json_config_to_registry_helper(self, name, sources): @@ -726,3 +737,279 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"])) self.assertEqual(f.read(), 'conflicting-upstream\n', "re-cloned layer must contain the upstream content after conflict backup") del os.environ['BBPATH'] + +@unittest.skipIf(os.environ.get("BB_SKIP_PYPI_TESTS", "yes") != "no", + "PyPI packaging test (set BB_SKIP_PYPI_TESTS=no to run)") +class PyPIPackagingTest(unittest.TestCase): + """Tests for PyPI packaging of bitbake-setup. + + These tests build a wheel from source, install it in an isolated venv, + and verify the package works correctly. Skipped by default unless + BB_SKIP_PYPI_TESTS=no is set. + """ + + wheel_path = None + build_tempdir = None + workspace_dir = None + + @classmethod + def setUpClass(cls): + """Build wheel once for all tests in this class.""" + # Locate contrib/pypi directory + cls.pypi_dir = os.path.abspath( + os.path.join(os.path.dirname(__file__), '..', '..', '..', 'contrib', 'pypi') + ) + + # Check for required build tools + cls._check_build_tools() + + # Create temporary directory for packaging workspace + cls.build_tempdir = tempfile.mkdtemp(prefix="bitbake-pypi-build-") + + # Run package-bitbake-setup.py to create packaging workspace + cls._create_packaging_workspace() + + # Build the wheel + cls._build_wheel() + + @classmethod + def _check_build_tools(cls): + """Verify build tools are available, skip if not.""" + try: + result = subprocess.run( + [sys.executable, "-m", "build", "--version"], + capture_output=True, check=True + ) + except (subprocess.CalledProcessError, FileNotFoundError): + raise unittest.SkipTest("'build' package not installed (pip install build)") + + @classmethod + def _create_packaging_workspace(cls): + """Create packaging workspace using package-bitbake-setup.py.""" + script = os.path.join(cls.pypi_dir, 'package-bitbake-setup.py') + cls.workspace_dir = os.path.join(cls.build_tempdir, 'workspace') + result = subprocess.run( + [sys.executable, script, '-d', cls.workspace_dir], + capture_output=True, text=True + ) + if result.returncode != 0: + raise unittest.SkipTest(f"Packaging workspace creation failed: {result.stderr}") + + @classmethod + def _build_wheel(cls): + """Build the wheel in the packaging workspace.""" + result = subprocess.run( + [sys.executable, "-m", "build", "--wheel"], + cwd=cls.workspace_dir, + capture_output=True, text=True + ) + if result.returncode != 0: + raise unittest.SkipTest(f"Wheel build failed: {result.stderr}") + + # Find the built wheel + dist_dir = os.path.join(cls.workspace_dir, 'dist') + wheels = glob.glob(os.path.join(dist_dir, '*.whl')) + if not wheels: + raise unittest.SkipTest("No wheel file found after build") + cls.wheel_path = wheels[0] + + @classmethod + def tearDownClass(cls): + """Clean up the shared wheel build artifacts.""" + if cls.build_tempdir and os.environ.get("BB_TMPDIR_NOCLEAN") != "yes": + shutil.rmtree(cls.build_tempdir, ignore_errors=True) + elif cls.build_tempdir: + print(f"Not cleaning up {cls.build_tempdir}. Please remove manually.") + + def setUp(self): + """Create isolated venv for testing the installed package.""" + self.venv_dir = tempfile.mkdtemp(prefix="bitbake-pypi-venv-") + + # Create venv without pip (faster, no network needed) + venv.create(self.venv_dir, with_pip=False, symlinks=True) + + # Get paths to venv python and bin directory + if sys.platform == 'win32': + self.venv_python = os.path.join(self.venv_dir, 'Scripts', 'python.exe') + self.venv_bin = os.path.join(self.venv_dir, 'Scripts') + else: + self.venv_python = os.path.join(self.venv_dir, 'bin', 'python') + self.venv_bin = os.path.join(self.venv_dir, 'bin') + + # Install wheel using pip from the outer environment (offline, no-deps) + site_packages = self._get_site_packages() + result = subprocess.run( + [sys.executable, "-m", "pip", "install", + "--target", site_packages, + "--no-deps", "--no-index", + self.wheel_path], + capture_output=True, text=True + ) + if result.returncode != 0: + self.fail(f"Failed to install wheel: {result.stderr}") + + # Install console script entry point manually + self._install_console_script() + + def _get_site_packages(self): + """Get the site-packages directory in the venv.""" + lib_dir = os.path.join(self.venv_dir, 'lib') + # Find python version directory + for d in os.listdir(lib_dir): + if d.startswith('python'): + return os.path.join(lib_dir, d, 'site-packages') + raise RuntimeError("Could not find site-packages in venv") + + def _install_console_script(self): + """Create console script wrapper in venv bin directory.""" + site_packages = self._get_site_packages() + script_path = os.path.join(self.venv_bin, 'bitbake-setup') + script_content = f'''#!{self.venv_python} +import sys +sys.path.insert(0, "{site_packages}") +from bitbake_setup.__main__ import main +sys.exit(main()) +''' + with open(script_path, 'w') as f: + f.write(script_content) + os.chmod(script_path, 0o755) + + def tearDown(self): + """Remove venv after test.""" + if os.environ.get("BB_TMPDIR_NOCLEAN") != "yes": + shutil.rmtree(self.venv_dir, ignore_errors=True) + else: + print(f"Not cleaning up {self.venv_dir}. Please remove manually.") + + def test_imports(self): + """Verify all expected modules can be imported from installed package.""" + + site_packages = self._get_site_packages() + imports = ['bb', 'bitbake_setup'] + for module in imports: + result = subprocess.run( + [self.venv_python, '-c', f'import sys; sys.path.insert(0, "{site_packages}"); import {module}'], + capture_output=True, text=True + ) + self.assertEqual(result.returncode, 0, + f"Failed to import {module}: {result.stderr}") + + def test_console_script_help(self): + """Verify bitbake-setup --help runs successfully.""" + script = os.path.join(self.venv_bin, 'bitbake-setup') + result = subprocess.run( + [script, '--help'], + capture_output=True, text=True + ) + self.assertEqual(result.returncode, 0, + f"bitbake-setup --help failed: {result.stderr}") + self.assertIn('usage:', result.stdout.lower()) + + def test_console_script_list(self): + """Verify bitbake-setup list runs successfully.""" + script = os.path.join(self.venv_bin, 'bitbake-setup') + result = subprocess.run( + [script, 'list'], + capture_output=True, text=True + ) + # List may return 0 even with no configurations + self.assertEqual(result.returncode, 0, + f"bitbake-setup list failed: {result.stderr}") + + def test_package_metadata(self): + """Verify package metadata is correctly set.""" + site_packages = self._get_site_packages() + code = ''' +import json +import sys +sys.path.insert(0, "{}") +from importlib.metadata import metadata +m = metadata("bitbake-setup") +print(json.dumps({{ + "name": m["Name"], + "version": m["Version"], + "requires_python": m.get("Requires-Python", ""), + "license": m.get("License", ""), +}})) +'''.format(site_packages) + result = subprocess.run( + [self.venv_python, '-c', code], + capture_output=True, text=True + ) + self.assertEqual(result.returncode, 0, + f"Failed to get metadata: {result.stderr}") + + meta = json.loads(result.stdout) + self.assertEqual(meta['name'], 'bitbake-setup') + self.assertIn('>=3.9', meta['requires_python']) + + def test_vendored_dependencies(self): + """Verify vendored dependencies (bs4, ply, progressbar, simplediff) are not bundled in package.""" + # Check that vendored packages do not exist in root of wheel + with zipfile.ZipFile(self.wheel_path, 'r') as whl: + names = whl.namelist() + + # Check for expected package directories + expected = ['bs4/', 'ply/', 'progressbar/', 'simplediff/'] + for pkg in expected: + found = any(n.startswith(pkg) for n in names) + self.assertFalse(found, + f"Unexpected vendored package '{pkg}' found in wheel") + + def test_version_from_wheel(self): + """Verify version is set correctly (not fallback 0.0.0 unless expected).""" + import re + # Extract version from wheel filename + wheel_name = os.path.basename(self.wheel_path) + # Wheel format: {name}-{version}(-{build})?-{python}-{abi}-{platform}.whl + parts = wheel_name.split('-') + version = parts[1] + + # Check version format (should be semver-like or contain git info) + version_pattern = r'^\d+\.\d+\.\d+.*$' + self.assertTrue(re.match(version_pattern, version), + f"Version '{version}' doesn't match expected pattern") + + print(f"Extracted version from wheel: {version}") + + self.assertNotEqual(version, '0.0.0', + "Version is fallback 0.0.0 - no git tags found") + + def test_wheel_metadata_file(self): + """Verify wheel METADATA file contains required fields.""" + with zipfile.ZipFile(self.wheel_path, 'r') as whl: + # Find METADATA file in dist-info + metadata_path = None + for name in whl.namelist(): + if name.endswith('.dist-info/METADATA'): + metadata_path = name + break + + self.assertIsNotNone(metadata_path, "METADATA file not found in wheel") + + # Parse metadata + metadata_content = whl.read(metadata_path).decode('utf-8') + + # Check required fields + self.assertIn('Metadata-Version:', metadata_content) + self.assertIn('Name: bitbake-setup', metadata_content) + self.assertIn('Version:', metadata_content) + self.assertIn('Requires-Python:', metadata_content) + + def test_entry_points(self): + """Verify console script entry points are correctly defined.""" + with zipfile.ZipFile(self.wheel_path, 'r') as whl: + # Find entry_points.txt in dist-info + entry_points_path = None + for name in whl.namelist(): + if name.endswith('.dist-info/entry_points.txt'): + entry_points_path = name + break + + self.assertIsNotNone(entry_points_path, + "entry_points.txt not found in wheel") + + content = whl.read(entry_points_path).decode('utf-8') + self.assertIn('[console_scripts]', content) + self.assertIn('bitbake-setup', content) + self.assertIn('bitbake_setup.__main__:main', content) From patchwork Tue Mar 31 14:53:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Woolley X-Patchwork-Id: 84916 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25BD1109B487 for ; Tue, 31 Mar 2026 14:53:33 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.22109.1774968805104877460 for ; Tue, 31 Mar 2026 07:53:25 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@windriver.com header.s=PPS06212021 header.b=sAHqhg6h; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=85502649bd=rob.woolley@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62V4w3D8274792; Tue, 31 Mar 2026 14:53:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=cc:content-transfer-encoding:content-type:date:from :in-reply-to:message-id:mime-version:references:subject:to; s= PPS06212021; bh=zncqOZQopmUy4dmIyKgDegLN3Y9jRAiSJvcTPJLHfUo=; b= sAHqhg6hj9ph+cFoVPsiYme1+g9kymt2HsouAIHiP91DoTvB1CSUxR4i76Fujm41 6+YGYT4tYR58K9AyTNcrttZ3OAq0equlcKwN83MMCEh1Wuj8rX8Hqae1RJWEqwqK kbOTwvLXYeZvoGiUpGptyvlh4gvFIK2T47W70cJEBsWZLzvAoiaGo4b6OnFRkxGF ML6rHBiE7X2GRWWTLWKoTMsYTa8zVASaPwKPFBdbpLbuagZZ/nbNAZYs/kQ7pN3P yB/J/9ACJ13WWw2tHxGlUTlQdP2d1XjYV5nuNcD+BbYCGDhFUWyA4/pQ/onMdpds ZDwZmvLyPP+SGvUzFr5YrQ== Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [128.224.246.36]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4d65y4btqq-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 31 Mar 2026 14:53:23 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (10.11.224.121) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.61; Tue, 31 Mar 2026 07:53:20 -0700 Received: from ala-lpggp3.wrs.com (10.11.232.110) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server id 15.1.2507.61 via Frontend Transport; Tue, 31 Mar 2026 07:53:20 -0700 From: Rob Woolley To: CC: Subject: [PATCH 3/6] pypi: Add packaging documentation for developers Date: Tue, 31 Mar 2026 07:53:16 -0700 Message-ID: <20260331145319.3125456-4-rob.woolley@windriver.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20260331145319.3125456-1-rob.woolley@windriver.com> References: <20260331145319.3125456-1-rob.woolley@windriver.com> MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=QaZrf8bv c=1 sm=1 tr=0 ts=69cbdfe3 cx=c_pps a=AbJuCvi4Y3V6hpbCNWx0WA==:117 a=AbJuCvi4Y3V6hpbCNWx0WA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=bi6dqmuHe4P4UrxVR6um:22 a=fTW__CHxibyLmBMfj2wP:22 a=t7CeM3EgAAAA:8 a=SIW4pjiE_UkdO4QYDIoA:9 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzMxMDE0NCBTYWx0ZWRfXzojbzNQ6ak8P jnq2/gOWIq8+lx2gxIDLZgQ9XkrHoDCUkaKb2KRjAc5vDyuo++4vkWkFzXuYe9cLAdy+GO89Mrj EPo2Kk3yrbRNLZyl3++ZPboLXlBk7NVg4Fo8ic6v/mbSL44CWhtew79O3HGYmiNGK/46LY8jY7n TL6rekefrBE06zx56nqnYyV+srjJG+TLYkGRaaOKUdqSZIV2fQLusBd9KJhuIV6pSOlBj+uIIPd Ri0vTHrARjh+RlRjnwJJ75BeLt3dazEL6iqka3laHZvW6Arsthsu+7VakGYJ+/7g+axB4QiQv3T NtjdpYL8bdG8IrF/HX1vTVCmyGZWlY0lF4jaPxs4RwBIMNPi0Y7/goVZ1Z64jiD1w0atd1S1emX jTu0YlUpHDf/EnIGdkFVXu0k8rblNPtYGRfdgMUmBLOm7pf9B8MQW14/mjk9Trug1hf39hyxTkk X9IxVwpAbleCbDCBRAw== X-Proofpoint-GUID: Xz4GwiEc3v-mcpQ5j8ZEjmyP-7aNoNSC X-Proofpoint-ORIG-GUID: Xz4GwiEc3v-mcpQ5j8ZEjmyP-7aNoNSC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-31_03,2026-03-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 adultscore=0 suspectscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603310144 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 31 Mar 2026 14:53:33 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19295 Signed-off-by: Rob Woolley --- contrib/pypi/BUILD.md | 50 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 contrib/pypi/BUILD.md diff --git a/contrib/pypi/BUILD.md b/contrib/pypi/BUILD.md new file mode 100644 index 00000000..b12fc083 --- /dev/null +++ b/contrib/pypi/BUILD.md @@ -0,0 +1,50 @@ +# Development Instructions + +## Requirements + +- Python >= 3.9 +- pip >= 19 (for installation) + +## Testing + +To lint the `bitbake-setup` pypi packaging, run the ruff tool. +```bash +ruff check bin/bitbake-setup contrib/pypi +``` + +The steps to build and test the `bitbake-setup` pypi packaging have been automated with the `bitbake-selftest` tool. This tool automatically creates a Python virtual environment for you. + +Run the bitbake-selftest +```bash +BB_SKIP_PYPI_TESTS=no bin/bitbake-selftest -v bb.tests.setup.PyPIPackagingTest +``` + +## Packaging + +### Create the development sandbox + +To create the development sandbox run: +```bash +contrib/pypi/package-bitbake-setup.py +cd packaging_workshop +``` + +### Building the package + +To install the development tools manually run: +```bash +python3 -m pip install -e '.[dev]' +``` + +To build a wheel (.whl) then use: +```bash +python3 -m build +``` + +This produces a wheel (.whl) file in the dist directory. This may be installed using pip. + +### Installing the package + +```bash +python3 -m pip install dist/bitbake_setup-*-py3-none-any.whl +``` From patchwork Tue Mar 31 14:53:17 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Woolley X-Patchwork-Id: 84915 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDBAC109B483 for ; Tue, 31 Mar 2026 14:53:32 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.22111.1774968806728137434 for ; Tue, 31 Mar 2026 07:53:26 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@windriver.com header.s=PPS06212021 header.b=XJ+wPAI1; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=85502649bd=rob.woolley@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62V4w3D9274792; Tue, 31 Mar 2026 14:53:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=cc:content-transfer-encoding:content-type:date:from :in-reply-to:message-id:mime-version:references:subject:to; s= PPS06212021; bh=aN+51oGesFcQGK4KG1xxQi0m3ns9ePeHgNSPKA6K/kY=; b= XJ+wPAI1RovKfHpHwbIsSuMu91RCTmgknjLhFp671dLGSIoIt9QQ8m2bdT0ytFD8 xy/ak+jX3ROU5CRUF8f2bgakykAyxo9C5fhHNv1/0nIQh9obwZPUPt2QGffILwV+ b4na5fRd4laVtFIx8E8FFUHudt4WZVP12OLKOBg+9rmrZPrCOrkPBpSrxGIFq+cI VyW3o6pA5TYEJ1zQFt7KDQFtxh0xAydLa8tegwBfWii51j1G7M9/X5tGP0lQihyj ZDN3Fx+4KzGPvTURCHKR5OYqoKNyDOerfAlsCYwPyIGpmITIbECRA1xzy9V4dgxo SqglGERuMg8bMFTKa9OvaA== Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [128.224.246.36]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4d65y4btqq-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 31 Mar 2026 14:53:24 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (10.11.224.121) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.61; Tue, 31 Mar 2026 07:53:20 -0700 Received: from ala-lpggp3.wrs.com (10.11.232.110) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server id 15.1.2507.61 via Frontend Transport; Tue, 31 Mar 2026 07:53:20 -0700 From: Rob Woolley To: CC: Subject: [PATCH 4/6] gitignore: Ignore temporary staging directory Date: Tue, 31 Mar 2026 07:53:17 -0700 Message-ID: <20260331145319.3125456-5-rob.woolley@windriver.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20260331145319.3125456-1-rob.woolley@windriver.com> References: <20260331145319.3125456-1-rob.woolley@windriver.com> MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=QaZrf8bv c=1 sm=1 tr=0 ts=69cbdfe4 cx=c_pps a=AbJuCvi4Y3V6hpbCNWx0WA==:117 a=AbJuCvi4Y3V6hpbCNWx0WA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=bi6dqmuHe4P4UrxVR6um:22 a=fTW__CHxibyLmBMfj2wP:22 a=t7CeM3EgAAAA:8 a=eOszy7m5RXGtjJtd4NEA:9 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzMxMDE0NCBTYWx0ZWRfX9ywx0AiXPM67 vA3WKi/dD1IZ7eGbAc1J/G61CNgVZ31fW2rouP3yC7AuGyuoQjBs4s/xn+81uH+MwY9BnZ5nIaO BiVIo0cl49mQuo+dMO1pDXZL1KjhcaWXoSSXXA/+62CngHuu7H3ZwKQleITPoJyqfbG+EY3QpSQ f5OYaGxiUyP5W4aXoqJg8CqG8VJWUhVLicdcelPW6b5yds8MhsJJ4hoHTaOuXFpBFn5QRVCiNA5 eLAT9T4bwZTOsGt4BSlLHbiV2/79NNWCexSW3pgFm3QZi4dxk5keAwjqMuIgBzMHDqVrEhYZc3Q mPysP3q9De8+s1iyUQUVtCyotVOcsgSoFPiAyRwVYumrkxoz98+0DhRBFwCxoPaGmne19nC3qJQ Aoolcu89aPmVQjQIYTpqjVdDtHonyaaODF7DV0z7p4mZBCB0LfEFd9a7zut/yZKHFHSX0U4i65Q 8u5bmr8lrinNmMYQG6Q== X-Proofpoint-GUID: DWJptpe2yq8iqWUv2k57E05gKJjqeuGw X-Proofpoint-ORIG-GUID: DWJptpe2yq8iqWUv2k57E05gKJjqeuGw X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-31_03,2026-03-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 adultscore=0 suspectscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603310144 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 31 Mar 2026 14:53:32 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19296 This directory gets created automatically by package-bitbake-setup.py when packaging bitbake-setup for PyPI. It is a temporary staging directory and should not be added to git. Signed-off-by: Rob Woolley --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index a6a256b2..9a5c4ec4 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,4 @@ lib/toaster/contrib/tts/backlog.txt lib/toaster/contrib/tts/log/* lib/toaster/contrib/tts/.cache/* lib/bb/tests/runqueue-tests/bitbake-cookerdaemon.log +packaging_workspace/ From patchwork Tue Mar 31 14:53:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Woolley X-Patchwork-Id: 84919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E08A109B485 for ; Tue, 31 Mar 2026 14:53:33 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.22113.1774968807904257196 for ; Tue, 31 Mar 2026 07:53:28 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@windriver.com header.s=PPS06212021 header.b=XRfgI20S; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=85502649bd=rob.woolley@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62V4w3DA274792; Tue, 31 Mar 2026 14:53:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=cc:content-transfer-encoding:content-type:date:from :in-reply-to:message-id:mime-version:references:subject:to; s= PPS06212021; bh=0d5CNGxg8JeZjZigH0QTLLBiQgeFSRwfQxs//03MdcI=; b= XRfgI20Sz1JKGfOv4PNcGPS9LAD1u6pzpawodKPloDjhS99+GBOPEwY9qtL6DYth zpluwWv8pBvOX43x6/YZvS68EEkTTVohGN1yzr9WnRjujbPqoWzHM3ptXeaAFNw6 ZVjK0/1JAdNWfhOimiPPZzALB9LUJs2SXv9DoM//IKjE53DJ5qgnz+vGnaeKSY2A fpqJQt7x3C329PMZqnBiokGVzznelxZEta1/F6TOLsDRuxHQ6U60ajz5KH3Ra737 zDP7+CIXBFbuOL4rCWVybt6K7G2vjFJ6rc9x0aEu47aZvnAMYHKh6Isoo/fJf52z 3opD8AYjVD2XpDb0qD5qPw== Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [128.224.246.36]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4d65y4btqq-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 31 Mar 2026 14:53:25 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (10.11.224.121) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.61; Tue, 31 Mar 2026 07:53:20 -0700 Received: from ala-lpggp3.wrs.com (10.11.232.110) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server id 15.1.2507.61 via Frontend Transport; Tue, 31 Mar 2026 07:53:20 -0700 From: Rob Woolley To: CC: Subject: [PATCH 5/6] lib: Vendorize bundled third-party libraries under bb._vendor Date: Tue, 31 Mar 2026 07:53:18 -0700 Message-ID: <20260331145319.3125456-6-rob.woolley@windriver.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20260331145319.3125456-1-rob.woolley@windriver.com> References: <20260331145319.3125456-1-rob.woolley@windriver.com> MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=QaZrf8bv c=1 sm=1 tr=0 ts=69cbdfe6 cx=c_pps a=AbJuCvi4Y3V6hpbCNWx0WA==:117 a=AbJuCvi4Y3V6hpbCNWx0WA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=bi6dqmuHe4P4UrxVR6um:22 a=fTW__CHxibyLmBMfj2wP:22 a=t7CeM3EgAAAA:8 a=sdMhlzqdCMuPHYe87FcA:9 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzMxMDE0NCBTYWx0ZWRfX+Xym1mPDSdnD QHY3FDJcEFHxTIab9WpXNa46q120c1VfONHSoGLgIVBgimTGF+6PtCDew+wdT3BO0uEY5z0PwYL y6Wm5jw7GV/Trd2C9Lq3V1839rCPwYGxdiIEH/n2FFq4sdoC9dmbQ6kEAfA4LooHQLHQ5CgDRjr tnXd/rS8sAgP94kL1EEoJVwZpub++cDXdgLn4XFIARfFQ/zX9Yw1hTNOjBDp0jNadxc1H+yDFcg UD9SLZ3Cw2K8ktQP3PTHxsY3bbL/dSEioHKrEEWYLkjzdhXTsgm63ohHy77W+fIdnRar7LlA0YL +u450cj+lHhVtXmFJ1qW0phexIWCzL/gld/QESUK/HqjTB5fNQPJdnULvJOnH8QRY0Nm+P1KgZk igR5hdMRTA/S7VFSZKUk0BqgXXWmEbmGIRKw8mvD9NzbnekddYYg3cWZqU7ePZBxURPAYzJZ76q FczM4eU97iOI8QJwVBw== X-Proofpoint-GUID: EP4UDBAAKUcXoF5NxupBPzTcT0AaisKr X-Proofpoint-ORIG-GUID: EP4UDBAAKUcXoF5NxupBPzTcT0AaisKr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-31_03,2026-03-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 adultscore=0 suspectscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603310144 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 31 Mar 2026 14:53:33 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19297 Move ply, progressbar, simplediff, and bs4 from lib/ into lib/bb/_vendor/ to avoid conflicts with system-installed versions and to support proper packaging. These libraries contain local modifications from their upstream versions. - Create lib/bb/_vendor/__init__.py - Move ply/, progressbar/, simplediff/, bs4/ into bb/_vendor/ - Convert absolute self-references to relative imports within ply (yacc.py) and bs4 (20 import sites across 8 files) - Update all external import sites to use bb._vendor prefix: bb/pysh/pyshlex.py, bb/pysh/pyshyacc.py, bb/ui/knotty.py, bb/siggen.py, bb/fetch2/wget.py, toaster/tests/views/test_views.py Assisted-by: Claude:claude-4.6-opus Signed-off-by: Rob Woolley --- lib/bb/_vendor/__init__.py | 18 ++++++++++++++++++ lib/{ => bb/_vendor}/bs4/AUTHORS | 0 lib/{ => bb/_vendor}/bs4/CHANGELOG | 0 lib/{ => bb/_vendor}/bs4/LICENSE | 0 lib/{ => bb/_vendor}/bs4/__init__.py | 0 lib/{ => bb/_vendor}/bs4/builder/__init__.py | 2 +- lib/{ => bb/_vendor}/bs4/builder/_html5lib.py | 12 ++++++------ .../_vendor}/bs4/builder/_htmlparser.py | 6 +++--- lib/{ => bb/_vendor}/bs4/builder/_lxml.py | 6 +++--- lib/{ => bb/_vendor}/bs4/css.py | 2 +- lib/{ => bb/_vendor}/bs4/dammit.py | 0 lib/{ => bb/_vendor}/bs4/diagnose.py | 9 ++++----- lib/{ => bb/_vendor}/bs4/element.py | 6 +++--- lib/{ => bb/_vendor}/bs4/formatter.py | 2 +- lib/{ => bb/_vendor}/ply/__init__.py | 0 lib/{ => bb/_vendor}/ply/lex.py | 0 lib/{ => bb/_vendor}/ply/yacc.py | 6 +----- lib/{ => bb/_vendor}/progressbar/LICENSE.txt | 0 lib/{ => bb/_vendor}/progressbar/__init__.py | 0 lib/{ => bb/_vendor}/progressbar/compat.py | 0 .../_vendor}/progressbar/progressbar.py | 0 lib/{ => bb/_vendor}/progressbar/widgets.py | 0 lib/{ => bb/_vendor}/simplediff/LICENSE | 0 lib/{ => bb/_vendor}/simplediff/__init__.py | 0 lib/bb/fetch2/wget.py | 4 ++-- lib/bb/pysh/pyshlex.py | 2 +- lib/bb/pysh/pyshyacc.py | 2 +- lib/bb/siggen.py | 2 +- lib/bb/ui/knotty.py | 2 +- lib/toaster/tests/views/test_views.py | 2 +- 30 files changed, 48 insertions(+), 35 deletions(-) create mode 100644 lib/bb/_vendor/__init__.py rename lib/{ => bb/_vendor}/bs4/AUTHORS (100%) rename lib/{ => bb/_vendor}/bs4/CHANGELOG (100%) rename lib/{ => bb/_vendor}/bs4/LICENSE (100%) rename lib/{ => bb/_vendor}/bs4/__init__.py (100%) rename lib/{ => bb/_vendor}/bs4/builder/__init__.py (99%) rename lib/{ => bb/_vendor}/bs4/builder/_html5lib.py (98%) rename lib/{ => bb/_vendor}/bs4/builder/_htmlparser.py (99%) rename lib/{ => bb/_vendor}/bs4/builder/_lxml.py (99%) rename lib/{ => bb/_vendor}/bs4/css.py (99%) rename lib/{ => bb/_vendor}/bs4/dammit.py (100%) rename lib/{ => bb/_vendor}/bs4/diagnose.py (96%) rename lib/{ => bb/_vendor}/bs4/element.py (99%) rename lib/{ => bb/_vendor}/bs4/formatter.py (99%) rename lib/{ => bb/_vendor}/ply/__init__.py (100%) rename lib/{ => bb/_vendor}/ply/lex.py (100%) rename lib/{ => bb/_vendor}/ply/yacc.py (99%) rename lib/{ => bb/_vendor}/progressbar/LICENSE.txt (100%) rename lib/{ => bb/_vendor}/progressbar/__init__.py (100%) rename lib/{ => bb/_vendor}/progressbar/compat.py (100%) rename lib/{ => bb/_vendor}/progressbar/progressbar.py (100%) rename lib/{ => bb/_vendor}/progressbar/widgets.py (100%) rename lib/{ => bb/_vendor}/simplediff/LICENSE (100%) rename lib/{ => bb/_vendor}/simplediff/__init__.py (100%) diff --git a/lib/bb/_vendor/__init__.py b/lib/bb/_vendor/__init__.py new file mode 100644 index 00000000..3c054dc3 --- /dev/null +++ b/lib/bb/_vendor/__init__.py @@ -0,0 +1,18 @@ +# +# Copyright BitBake Contributors +# +# SPDX-License-Identifier: GPL-2.0-only +# + +""" +Vendored third-party libraries for BitBake. + +These libraries have been modified from their upstream versions and are +bundled here to avoid conflicts with system-installed packages. + +Vendored packages: + - bs4 (BeautifulSoup4) + - ply + - progressbar + - simplediff +""" diff --git a/lib/bs4/AUTHORS b/lib/bb/_vendor/bs4/AUTHORS similarity index 100% rename from lib/bs4/AUTHORS rename to lib/bb/_vendor/bs4/AUTHORS diff --git a/lib/bs4/CHANGELOG b/lib/bb/_vendor/bs4/CHANGELOG similarity index 100% rename from lib/bs4/CHANGELOG rename to lib/bb/_vendor/bs4/CHANGELOG diff --git a/lib/bs4/LICENSE b/lib/bb/_vendor/bs4/LICENSE similarity index 100% rename from lib/bs4/LICENSE rename to lib/bb/_vendor/bs4/LICENSE diff --git a/lib/bs4/__init__.py b/lib/bb/_vendor/bs4/__init__.py similarity index 100% rename from lib/bs4/__init__.py rename to lib/bb/_vendor/bs4/__init__.py diff --git a/lib/bs4/builder/__init__.py b/lib/bb/_vendor/bs4/builder/__init__.py similarity index 99% rename from lib/bs4/builder/__init__.py rename to lib/bb/_vendor/bs4/builder/__init__.py index ffb31fc2..d60a9a67 100644 --- a/lib/bs4/builder/__init__.py +++ b/lib/bb/_vendor/bs4/builder/__init__.py @@ -6,7 +6,7 @@ import itertools import re import warnings import sys -from bs4.element import ( +from ..element import ( CharsetMetaAttributeValue, ContentMetaAttributeValue, RubyParenthesisString, diff --git a/lib/bs4/builder/_html5lib.py b/lib/bb/_vendor/bs4/builder/_html5lib.py similarity index 98% rename from lib/bs4/builder/_html5lib.py rename to lib/bb/_vendor/bs4/builder/_html5lib.py index 7c46a851..8ca19fec 100644 --- a/lib/bs4/builder/_html5lib.py +++ b/lib/bb/_vendor/bs4/builder/_html5lib.py @@ -7,14 +7,14 @@ __all__ = [ import warnings import re -from bs4.builder import ( +from . import ( DetectsXMLParsedAsHTML, PERMISSIVE, HTML, HTML_5, HTMLTreeBuilder, ) -from bs4.element import ( +from ..element import ( NamespacedAttribute, nonwhitespace_re, ) @@ -23,7 +23,7 @@ from html5lib.constants import ( namespaces, prefixes, ) -from bs4.element import ( +from ..element import ( Comment, Doctype, NavigableString, @@ -134,7 +134,7 @@ class TreeBuilderForHtml5lib(treebuilder_base.TreeBuilder): if soup: self.soup = soup else: - from bs4 import BeautifulSoup + from .. import BeautifulSoup # TODO: Why is the parser 'html.parser' here? To avoid an # infinite loop? self.soup = BeautifulSoup( @@ -180,7 +180,7 @@ class TreeBuilderForHtml5lib(treebuilder_base.TreeBuilder): return TextNode(Comment(data), self.soup) def fragmentClass(self): - from bs4 import BeautifulSoup + from .. import BeautifulSoup # TODO: Why is the parser 'html.parser' here? To avoid an # infinite loop? self.soup = BeautifulSoup("", "html.parser") @@ -198,7 +198,7 @@ class TreeBuilderForHtml5lib(treebuilder_base.TreeBuilder): return treebuilder_base.TreeBuilder.getFragment(self).element def testSerializer(self, element): - from bs4 import BeautifulSoup + from .. import BeautifulSoup rv = [] doctype_re = re.compile(r'^(.*?)(?: PUBLIC "(.*?)"(?: "(.*?)")?| SYSTEM "(.*?)")?$') diff --git a/lib/bs4/builder/_htmlparser.py b/lib/bb/_vendor/bs4/builder/_htmlparser.py similarity index 99% rename from lib/bs4/builder/_htmlparser.py rename to lib/bb/_vendor/bs4/builder/_htmlparser.py index 3cc187f8..f55cbadf 100644 --- a/lib/bs4/builder/_htmlparser.py +++ b/lib/bb/_vendor/bs4/builder/_htmlparser.py @@ -13,16 +13,16 @@ from html.parser import HTMLParser import sys import warnings -from bs4.element import ( +from ..element import ( CData, Comment, Declaration, Doctype, ProcessingInstruction, ) -from bs4.dammit import EntitySubstitution, UnicodeDammit +from ..dammit import EntitySubstitution, UnicodeDammit -from bs4.builder import ( +from . import ( DetectsXMLParsedAsHTML, ParserRejectedMarkup, HTML, diff --git a/lib/bs4/builder/_lxml.py b/lib/bb/_vendor/bs4/builder/_lxml.py similarity index 99% rename from lib/bs4/builder/_lxml.py rename to lib/bb/_vendor/bs4/builder/_lxml.py index 4f7cf746..fc80133b 100644 --- a/lib/bs4/builder/_lxml.py +++ b/lib/bb/_vendor/bs4/builder/_lxml.py @@ -14,14 +14,14 @@ except ImportError as e: from io import BytesIO from io import StringIO from lxml import etree -from bs4.element import ( +from ..element import ( Comment, Doctype, NamespacedAttribute, ProcessingInstruction, XMLProcessingInstruction, ) -from bs4.builder import ( +from . import ( DetectsXMLParsedAsHTML, FAST, HTML, @@ -30,7 +30,7 @@ from bs4.builder import ( ParserRejectedMarkup, TreeBuilder, XML) -from bs4.dammit import EncodingDetector +from ..dammit import EncodingDetector LXML = 'lxml' diff --git a/lib/bs4/css.py b/lib/bb/_vendor/bs4/css.py similarity index 99% rename from lib/bs4/css.py rename to lib/bb/_vendor/bs4/css.py index cd1fd2df..7cbeb83c 100644 --- a/lib/bs4/css.py +++ b/lib/bb/_vendor/bs4/css.py @@ -68,7 +68,7 @@ class CSS(object): result (a common mistake). """ # Import here to avoid circular import - from bs4.element import ResultSet + from .element import ResultSet return ResultSet(None, results) def compile(self, select, namespaces=None, flags=0, **kwargs): diff --git a/lib/bs4/dammit.py b/lib/bb/_vendor/bs4/dammit.py similarity index 100% rename from lib/bs4/dammit.py rename to lib/bb/_vendor/bs4/dammit.py diff --git a/lib/bs4/diagnose.py b/lib/bb/_vendor/bs4/diagnose.py similarity index 96% rename from lib/bs4/diagnose.py rename to lib/bb/_vendor/bs4/diagnose.py index 46927953..76d0be8f 100644 --- a/lib/bs4/diagnose.py +++ b/lib/bb/_vendor/bs4/diagnose.py @@ -6,9 +6,8 @@ __license__ = "MIT" import cProfile from io import BytesIO from html.parser import HTMLParser -import bs4 -from bs4 import BeautifulSoup, __version__ -from bs4.builder import builder_registry +from . import BeautifulSoup, __version__ +from .builder import builder_registry import os import pstats @@ -219,8 +218,8 @@ def profile(num_elements=100000, parser="lxml"): filename = filehandle.name data = rdoc(num_elements) - vars = dict(bs4=bs4, data=data, parser=parser) - cProfile.runctx('bs4.BeautifulSoup(data, parser)' , vars, vars, filename) + vars = dict(BeautifulSoup=BeautifulSoup, data=data, parser=parser) + cProfile.runctx('BeautifulSoup(data, parser)' , vars, vars, filename) stats = pstats.Stats(filename) # stats.strip_dirs() diff --git a/lib/bs4/element.py b/lib/bb/_vendor/bs4/element.py similarity index 99% rename from lib/bs4/element.py rename to lib/bb/_vendor/bs4/element.py index 0aefe734..38ca2dc2 100644 --- a/lib/bs4/element.py +++ b/lib/bb/_vendor/bs4/element.py @@ -9,8 +9,8 @@ import re import sys import warnings -from bs4.css import CSS -from bs4.formatter import ( +from .css import CSS +from .formatter import ( Formatter, HTMLFormatter, XMLFormatter, @@ -420,7 +420,7 @@ class PageElement(object): and not isinstance(new_child, NavigableString)): new_child = NavigableString(new_child) - from bs4 import BeautifulSoup + from . import BeautifulSoup if isinstance(new_child, BeautifulSoup): # We don't want to end up with a situation where one BeautifulSoup # object contains another. Insert the children one at a time. diff --git a/lib/bs4/formatter.py b/lib/bb/_vendor/bs4/formatter.py similarity index 99% rename from lib/bs4/formatter.py rename to lib/bb/_vendor/bs4/formatter.py index 9fa1b57c..50f775ae 100644 --- a/lib/bs4/formatter.py +++ b/lib/bb/_vendor/bs4/formatter.py @@ -1,4 +1,4 @@ -from bs4.dammit import EntitySubstitution +from .dammit import EntitySubstitution class Formatter(EntitySubstitution): """Describes a strategy to use when outputting a parse tree to a string. diff --git a/lib/ply/__init__.py b/lib/bb/_vendor/ply/__init__.py similarity index 100% rename from lib/ply/__init__.py rename to lib/bb/_vendor/ply/__init__.py diff --git a/lib/ply/lex.py b/lib/bb/_vendor/ply/lex.py similarity index 100% rename from lib/ply/lex.py rename to lib/bb/_vendor/ply/lex.py diff --git a/lib/ply/yacc.py b/lib/bb/_vendor/ply/yacc.py similarity index 99% rename from lib/ply/yacc.py rename to lib/bb/_vendor/ply/yacc.py index 529f85b0..0cd9b522 100644 --- a/lib/ply/yacc.py +++ b/lib/bb/_vendor/ply/yacc.py @@ -100,12 +100,8 @@ try: except AttributeError: MAXINT = sys.maxsize -# Python 2.x/3.0 compatibility. def load_ply_lex(): - if sys.version_info[0] < 3: - import lex - else: - import ply.lex as lex + from . import lex return lex # This object is a stand-in for a logging object created by the diff --git a/lib/progressbar/LICENSE.txt b/lib/bb/_vendor/progressbar/LICENSE.txt similarity index 100% rename from lib/progressbar/LICENSE.txt rename to lib/bb/_vendor/progressbar/LICENSE.txt diff --git a/lib/progressbar/__init__.py b/lib/bb/_vendor/progressbar/__init__.py similarity index 100% rename from lib/progressbar/__init__.py rename to lib/bb/_vendor/progressbar/__init__.py diff --git a/lib/progressbar/compat.py b/lib/bb/_vendor/progressbar/compat.py similarity index 100% rename from lib/progressbar/compat.py rename to lib/bb/_vendor/progressbar/compat.py diff --git a/lib/progressbar/progressbar.py b/lib/bb/_vendor/progressbar/progressbar.py similarity index 100% rename from lib/progressbar/progressbar.py rename to lib/bb/_vendor/progressbar/progressbar.py diff --git a/lib/progressbar/widgets.py b/lib/bb/_vendor/progressbar/widgets.py similarity index 100% rename from lib/progressbar/widgets.py rename to lib/bb/_vendor/progressbar/widgets.py diff --git a/lib/simplediff/LICENSE b/lib/bb/_vendor/simplediff/LICENSE similarity index 100% rename from lib/simplediff/LICENSE rename to lib/bb/_vendor/simplediff/LICENSE diff --git a/lib/simplediff/__init__.py b/lib/bb/_vendor/simplediff/__init__.py similarity index 100% rename from lib/simplediff/__init__.py rename to lib/bb/_vendor/simplediff/__init__.py diff --git a/lib/bb/fetch2/wget.py b/lib/bb/fetch2/wget.py index ca4959ab..aaa5027f 100644 --- a/lib/bb/fetch2/wget.py +++ b/lib/bb/fetch2/wget.py @@ -27,8 +27,8 @@ from bb.fetch2 import FetchMethod from bb.fetch2 import FetchError from bb.fetch2 import logger from bb.fetch2 import runfetchcmd -from bs4 import BeautifulSoup -from bs4 import SoupStrainer +from bb._vendor.bs4 import BeautifulSoup +from bb._vendor.bs4 import SoupStrainer class WgetProgressHandler(bb.progress.LineFilterProgressHandler): """ diff --git a/lib/bb/pysh/pyshlex.py b/lib/bb/pysh/pyshlex.py index a42c2944..3f99cd1b 100644 --- a/lib/bb/pysh/pyshlex.py +++ b/lib/bb/pysh/pyshlex.py @@ -14,7 +14,7 @@ # that hard to enable pull mode. import re -from ply import lex +from bb._vendor.ply import lex from bb.pysh.sherrors import * class NeedMore(Exception): diff --git a/lib/bb/pysh/pyshyacc.py b/lib/bb/pysh/pyshyacc.py index 924860a6..67e22c96 100644 --- a/lib/bb/pysh/pyshyacc.py +++ b/lib/bb/pysh/pyshyacc.py @@ -13,7 +13,7 @@ import sys import bb.pysh.pyshlex as pyshlex tokens = pyshlex.tokens -from ply import yacc +from bb._vendor.ply import yacc import bb.pysh.sherrors as sherrors class IORedirect: diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py index 985fa7e4..4d0aa4ac 100644 --- a/lib/bb/siggen.py +++ b/lib/bb/siggen.py @@ -12,7 +12,7 @@ import tempfile import pickle import bb.data import difflib -import simplediff +from bb._vendor import simplediff import json import types from contextlib import contextmanager diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py index 82531ef8..15025e8d 100644 --- a/lib/bb/ui/knotty.py +++ b/lib/bb/ui/knotty.py @@ -14,7 +14,7 @@ import io import os import sys import logging -import progressbar +from bb._vendor import progressbar import signal import bb.msg import time diff --git a/lib/toaster/tests/views/test_views.py b/lib/toaster/tests/views/test_views.py index e1adfcf8..5e64e819 100644 --- a/lib/toaster/tests/views/test_views.py +++ b/lib/toaster/tests/views/test_views.py @@ -27,7 +27,7 @@ import toastergui from toastergui.tables import SoftwareRecipesTable import json -from bs4 import BeautifulSoup +from bb._vendor.bs4 import BeautifulSoup import string PROJECT_NAME = "test project" From patchwork Tue Mar 31 14:53:19 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Woolley X-Patchwork-Id: 84917 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AB04109B489 for ; Tue, 31 Mar 2026 14:53:33 +0000 (UTC) Received: from mx0b-0064b401.pphosted.com (mx0b-0064b401.pphosted.com [205.220.178.238]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.22114.1774968808601023264 for ; Tue, 31 Mar 2026 07:53:28 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@windriver.com header.s=PPS06212021 header.b=PEqgHTwS; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.178.238, mailfrom: prvs=85502649bd=rob.woolley@windriver.com) Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62V4w3DB274792; Tue, 31 Mar 2026 14:53:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=cc:content-transfer-encoding:content-type:date:from :in-reply-to:message-id:mime-version:references:subject:to; s= PPS06212021; bh=hZ0vSibT4yXa8xE8eqe219UrsfmVHoUvbrMpoONcMRg=; b= PEqgHTwSDt3MIy3h+BZflFfoX+38S20vmhwxui41z4CdmSdwXhnlMZ5/HwmZY7IY FXqJZU6wmxFWro1a1LmXc49MgGSrCEdrMy/S4OVxmFcQvz066zWTOPqloPFz3Kug 2qOXRaIFWtaZpS4YRAhu6sjJ9GV27ebKWIWid0qdR/7KTuJkOKDKhM8O1G6Ieij5 rm9Tx/82O0RR7/jmQwO/erwN2qkYht9jEp+RYTOKvdyWxklpq8gCpUfSLTfdBImC gllNXcT0W2yEOwizmgaoIxK7WwOHgPreUoobDkq5pOMkecrNwRk7Dsqu9fm+32c9 oHoeBUO0PnRjM/O/zUDOlQ== Received: from ala-exchng01.corp.ad.wrs.com (ala-exchng01.wrs.com [128.224.246.36]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4d65y4btqq-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 31 Mar 2026 14:53:27 +0000 (GMT) Received: from ala-exchng01.corp.ad.wrs.com (10.11.224.121) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.61; Tue, 31 Mar 2026 07:53:20 -0700 Received: from ala-lpggp3.wrs.com (10.11.232.110) by ala-exchng01.corp.ad.wrs.com (10.11.224.121) with Microsoft SMTP Server id 15.1.2507.61 via Frontend Transport; Tue, 31 Mar 2026 07:53:20 -0700 From: Rob Woolley To: CC: Subject: [PATCH 6/6] bb: Move codegen.py inside module Date: Tue, 31 Mar 2026 07:53:19 -0700 Message-ID: <20260331145319.3125456-7-rob.woolley@windriver.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20260331145319.3125456-1-rob.woolley@windriver.com> References: <20260331145319.3125456-1-rob.woolley@windriver.com> MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=QaZrf8bv c=1 sm=1 tr=0 ts=69cbdfe7 cx=c_pps a=AbJuCvi4Y3V6hpbCNWx0WA==:117 a=AbJuCvi4Y3V6hpbCNWx0WA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=bi6dqmuHe4P4UrxVR6um:22 a=fTW__CHxibyLmBMfj2wP:22 a=t7CeM3EgAAAA:8 a=s_keFEVr-WujLWSdrMAA:9 a=zZCYzV9kfG8A:10 a=FdTzh2GWekK77mhwV6Dw:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzMxMDE0NCBTYWx0ZWRfXzCg53Ipf8G0I tYd2PQ3iKMsDYK+9XU3TsB8TqL7GbA4++QXRVImiusI5HmArPfG+v1mncG890boE9T/672DsTS/ KCpx9kIxCoSS+ZvueFbf5q8Amw58DssAWsYb/Zu/zmE3Lw69FeA31uRc0X6KrTJbYXmRn+uMwot QUwdr8IcugtSgPEVcP4nOo4VIC2q0ESGyumEMa2F3Bek3Zb2TcqlgKrRCYbidPQIhj6CRNsdq9a V23HWb1UGUwu3sNLenQFsWi4bnRRK/AJ2wadii3BhJC3HbhtKmazQY7kWeSDR2fYNW+CY41oFsv elyr1bnyE5q6wIhN5/SgBbieGFAmgc8HQdpHnqZWppVq/K0+YC1ESjtSzYj96+wk8qMOMAqXFz8 v2w2U3A/u8uVemD8TQR9qpb+X2HvVTGllzOyWVio5J2qOq+qeHo1j6QEcg52vYzFS55weXq1L7b mK8SykN90pkfUgwtPVA== X-Proofpoint-GUID: HhVnSE1GD1T0X6FdkNned2S6oqMCT09x X-Proofpoint-ORIG-GUID: HhVnSE1GD1T0X6FdkNned2S6oqMCT09x X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-31_03,2026-03-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 adultscore=0 suspectscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603310144 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 31 Mar 2026 14:53:33 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19298 Codegen is used exclusively by bb.codeparser.PythonParser. Move it inside the bb module to avoid exposing it externally when installing bb as a package. Signed-off-by: Rob Woolley --- lib/{ => bb}/codegen.py | 0 lib/bb/codeparser.py | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) rename lib/{ => bb}/codegen.py (100%) diff --git a/lib/codegen.py b/lib/bb/codegen.py similarity index 100% rename from lib/codegen.py rename to lib/bb/codegen.py diff --git a/lib/bb/codeparser.py b/lib/bb/codeparser.py index 4f70cf7f..76e4eea2 100644 --- a/lib/bb/codeparser.py +++ b/lib/bb/codeparser.py @@ -25,9 +25,9 @@ recipe caches don't trigger "Taskhash mismatch" errors. import ast import sys -import codegen import logging import inspect +import bb.codegen import bb.pysh as pysh import bb.utils, bb.data import hashlib