| Message ID | 20260121024723.2634264-5-rob.woolley@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | Package bitbake-setup for PyPI | expand |
On Wed, 21 Jan 2026 at 03:47, Rob Woolley via lists.openembedded.org
<rob.woolley=windriver.com@lists.openembedded.org> wrote:
> +version = 2.16.0a1
There are other issues in setup.cfg, and the commit title should be:
'bitbake-setup: add pypi packaging', but before that I think we need
to consider the release strategy and version scheme for it, and get
RP's blessing :)
Here's what there is in terms of tags and branches, and from what I've
heard it is not easy to change:
https://git.openembedded.org/bitbake/
I guess, bitbake-setup would only be released from the master branch,
at least once when a new yocto release happens, at the point where
stable and master branches diverge. Those releases should match the
stable version, e.g. whinlatter release would have been 2.16.
Then the question I can't quite answer is, how to version something
from the master trunk? E.g. if we want to release from master right
now, what version would it have? Note that ideally it needs to contain
the short commit id, for traceability.
Alex
Hi Rob, On 1/21/26 3:47 AM, Rob Woolley via lists.openembedded.org wrote: > This enables creating a pip package for PyPI with: > > python3 -m build > > Signed-off-by: Rob Woolley <rob.woolley@windriver.com> > --- > pyproject.toml | 3 +++ > setup.cfg | 40 ++++++++++++++++++++++++++++++++++++++++ Please migrate everything in setup.cfg into pyproject.toml, there's no reason to use an ancient mechanism. (This may require a more recent setuptools) > 2 files changed, 43 insertions(+) > create mode 100644 pyproject.toml > create mode 100644 setup.cfg > > diff --git a/pyproject.toml b/pyproject.toml > new file mode 100644 > index 00000000..4aff98bb > --- /dev/null > +++ b/pyproject.toml > @@ -0,0 +1,3 @@ > +[build-system] > +requires = ["setuptools>=61.0.0", "wheel"] > +build-backend = "setuptools.build_meta" > diff --git a/setup.cfg b/setup.cfg > new file mode 100644 > index 00000000..3eaa3c49 > --- /dev/null > +++ b/setup.cfg > @@ -0,0 +1,40 @@ > +[metadata] > +name = bitbake-setup > +version = 2.16.0a1 100% we are going to forget updating this field. I'm wondering if we cannot simply use setuptools_scm to have a dynamic version? Or use the version from lib/bb/__init__.py? > +author = OpenEmbedded BitBake Developers > +author_email = bitbake-devel@lists.openembedded.org > +description = bitbake-setup > +long_description = file: README.md > +long_description_content_type = text/markdown > +url = https://github.com/openembedded/bitbake.git > +license = "GPL-2.0-only and MIT" Must be uppercase, so AND. c.f. https://spdx.github.io/spdx-spec/v2.2.2/SPDX-license-expressions/#d2-case-sensitivity > +classifiers = > + Programming Language :: Python :: 3 > + License :: OSI Approved :: MIT License > + License :: OSI Approved :: GNU General Public License v2 (GPLv2) This is deprecated since PEP639, c.f. https://peps.python.org/pep-0639/#deprecate-license-classifiers > + Operating System :: POSIX :: Linux > + > +[options] > +package_dir = > + = lib > +packages = find: > +include_package_data = True > +zip_safe = False > +python_requires = >=3.6 3.9.0 as reported in __init__.py. > +py_modules = codegen > + > +install_requires = > + ; List any non-standard dependencies BitBake needs > + ; BitBake is mostly self-contained, but check its documentation > + > +[options.packages.find] > +where = lib > +include = > + bb* Is this meant to match both lib/bb/ and lib/bblayers/ ? Why not explicitly list both? > + bs4* What is this supposed to match except lib/bs4/? Cheers, Quentin
Hi Alex, Rob, On 1/21/26 10:41 AM, Alexander Kanavin via lists.openembedded.org wrote: > On Wed, 21 Jan 2026 at 03:47, Rob Woolley via lists.openembedded.org > <rob.woolley=windriver.com@lists.openembedded.org> wrote: >> +version = 2.16.0a1 > > There are other issues in setup.cfg, and the commit title should be: > 'bitbake-setup: add pypi packaging', but before that I think we need > to consider the release strategy and version scheme for it, and get > RP's blessing :) > > Here's what there is in terms of tags and branches, and from what I've > heard it is not easy to change: > https://git.openembedded.org/bitbake/ > > I guess, bitbake-setup would only be released from the master branch, > at least once when a new yocto release happens, at the point where > stable and master branches diverge. Those releases should match the > stable version, e.g. whinlatter release would have been 2.16. > > Then the question I can't quite answer is, how to version something > from the master trunk? E.g. if we want to release from master right > now, what version would it have? Note that ideally it needs to contain > the short commit id, for traceability. > setuptools_scm should be able to handle that, see https://setuptools-scm.readthedocs.io/en/stable/usage/#default-versioning-scheme. I've yet to play with it (will try with https://github.com/yoctoproject/bmaptool/ in the next few weeks). Note that as far as I could tell this means all versioned files in the repo are part of the sdist by default (but you can exclude them via MANIFEST.in). The next question is whether we're planning to upload bitbake itself to PyPI in the future? If so, we need to be careful about the layout. Building two packages from the same git repo isn't that straightforward if I remember correctly so if we're moving things around, better anticipate this possibility so we don't have to move everything around again once we want to publish bitbake itself. Cheers, Quentin
Hi, On Wed Jan 21, 2026 at 3:47 AM CET, Rob Woolley via lists.openembedded.org wrote: > This enables creating a pip package for PyPI with: > > python3 -m build > > Signed-off-by: Rob Woolley <rob.woolley@windriver.com> > --- > pyproject.toml | 3 +++ > setup.cfg | 40 ++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 43 insertions(+) > create mode 100644 pyproject.toml > create mode 100644 setup.cfg > > diff --git a/pyproject.toml b/pyproject.toml > new file mode 100644 > index 00000000..4aff98bb > --- /dev/null > +++ b/pyproject.toml > @@ -0,0 +1,3 @@ > +[build-system] > +requires = ["setuptools>=61.0.0", "wheel"] > +build-backend = "setuptools.build_meta" > diff --git a/setup.cfg b/setup.cfg > new file mode 100644 > index 00000000..3eaa3c49 > --- /dev/null > +++ b/setup.cfg > @@ -0,0 +1,40 @@ > +[metadata] > +name = bitbake-setup > +version = 2.16.0a1 > +author = OpenEmbedded BitBake Developers > +author_email = bitbake-devel@lists.openembedded.org > +description = bitbake-setup > +long_description = file: README.md I don't think we have a README.md file? Perhaps doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst would be a better candidate here? > +long_description_content_type = text/markdown This would be text/x-rst then. > +url = https://github.com/openembedded/bitbake.git It should probably be https://git.openembedded.org/bitbake/? The github version is supposed to be a mirror only I think. Antonin
On Wed, 21 Jan 2026 at 11:02, Quentin Schulz via lists.openembedded.org <quentin.schulz=cherry.de@lists.openembedded.org> wrote: > The next question is whether we're planning to upload bitbake itself to > PyPI in the future? If so, we need to be careful about the layout. > Building two packages from the same git repo isn't that straightforward > if I remember correctly so if we're moving things around, better > anticipate this possibility so we don't have to move everything around > again once we want to publish bitbake itself. The answer to publishing bitbake itself is a firm no. That said, bitbake-setup packaging files still shouldn't be at the top level. There should be a script that moves them into right place before doing the packaging step, or ideally packaging should be pointed to use a sub-directory. Alex
On Wed, Jan 21, 2026 at 5:02 AM Quentin Schulz via lists.openembedded.org <quentin.schulz=cherry.de@lists.openembedded.org> wrote: > On 1/21/26 10:41 AM, Alexander Kanavin via lists.openembedded.org wrote: > > On Wed, 21 Jan 2026 at 03:47, Rob Woolley via lists.openembedded.org > > <rob.woolley=windriver.com@lists.openembedded.org> wrote: > >> +version = 2.16.0a1 > > > > There are other issues in setup.cfg, and the commit title should be: > > 'bitbake-setup: add pypi packaging', but before that I think we need > > to consider the release strategy and version scheme for it, and get > > RP's blessing :) > > > > Here's what there is in terms of tags and branches, and from what I've > > heard it is not easy to change: > > https://git.openembedded.org/bitbake/ > > > > I guess, bitbake-setup would only be released from the master branch, > > at least once when a new yocto release happens, at the point where > > stable and master branches diverge. Those releases should match the > > stable version, e.g. whinlatter release would have been 2.16. > > > > Then the question I can't quite answer is, how to version something > > from the master trunk? E.g. if we want to release from master right > > now, what version would it have? Note that ideally it needs to contain > > the short commit id, for traceability. > > > > setuptools_scm should be able to handle that, see > > https://setuptools-scm.readthedocs.io/en/stable/usage/#default-versioning-scheme. > > I've yet to play with it (will try with > https://github.com/yoctoproject/bmaptool/ in the next few weeks). > > Note that as far as I could tell this means all versioned files in the > repo are part of the sdist by default (but you can exclude them via > MANIFEST.in). > > The next question is whether we're planning to upload bitbake itself to > PyPI in the future? If so, we need to be careful about the layout. > Building two packages from the same git repo isn't that straightforward > if I remember correctly so if we're moving things around, better > anticipate this possibility so we don't have to move everything around > again once we want to publish bitbake itself. > > Cheers, > Quentin > Quick update on progress: I have replaced nox and pytest with Python unittest so that the tests can be invoked from bitbake-selftest. The new PyPIPackagingTest still uses package-bitbake-setup.py to copy the files from contrib/pypi to a staging directory. This is needed to reorganize the layout such that pyproject.toml is at the root with the source code in subdirectories. I experimented with using setuptools_scm to automatically detect the version number for creating the Python package. Using setuptools_scm for determining the version based on git tags conflicts with creating the staging directory. Whether it is outside the git repo (e.g. in TMPDIR) or in a sub-directory (e.g. <git repo root>/packaging_workspace) it fails to detect the latest git tag. In the case of it being in a subdirectory, I need to set root (e.g. root = "..") in the tool.setuptools_scm section of pyproject.toml for it to succeed. If we want to use setuptools_scm with git for automatic versioning then we will need to use a fixed (temporary) directory in the git repository (eg. ./packaging_workspace). If a developer chooses another location for scratch work the version will fallback to 0.0.0, which seems acceptable to me. I added a test in PyPIPackagingTest to check for this failure case when doing "production" builds. Are we okay with pinning the temporary staging directory inside the git repository? If so, do you have a preference as to what that directory is called and where it is located? The setuptools_scm uses git tags rather than branches. We may want to consider updating our tagging strategy if we want to use it to automatically describe the commit for packages built on master. One option we have is to set tag_regex in pyproject.toml to a custom pattern. Thus we could create tags like bitbake-v2.16.0 or bitbake-setup-v2.16.0 to provide a reference point for results like bitbake-setup-2.16.1.dev0+gd2e159d85.d20260211 or something simpler like bitbake-setup-2.16.2.dev1 (see https://setuptools-scm.readthedocs.io/en/latest/extending/#version-number-construction ) Regards, Rob
On Wed, 11 Feb 2026 at 15:41, Rob Woolley <rob.woolleywr@gmail.com> wrote: > Are we okay with pinning the temporary staging directory inside the git repository? If so, do you have a preference as to what that directory is called and where it is located? I am okay and I have no particular preference. As long as it's in .gitignore. > The setuptools_scm uses git tags rather than branches. We may want to consider updating our tagging strategy if we want to use it to automatically describe the commit for packages built on master. One option we have is to set tag_regex in pyproject.toml to a custom pattern. Thus we could create tags like bitbake-v2.16.0 or bitbake-setup-v2.16.0 to provide a reference point for results like bitbake-setup-2.16.1.dev0+gd2e159d85.d20260211 or something simpler like bitbake-setup-2.16.2.dev1 (see https://setuptools-scm.readthedocs.io/en/latest/extending/#version-number-construction) The tags available in master branch can be see with 'git tag --merged master'. It seems that the two most suitable sets are at the end: 2024-04-scarthgap 2024-10-styhead 2025-04-walnascar 2025-04.1-walnascar 2025-10-whinlatter ... yocto-5.0 yocto-5.1 yocto-5.2 yocto-5.2.1 yocto-5.3 So we could simply use one of these versions (they match exact release points), and then add a .relN+hash if that can be done automatically. And then there's no need to introduce new tag schemes or extra work for RP. I like the one with the year and release name a bit more, because it's more informative to the users. Alex
Hi Alex, On 2/12/26 12:38 PM, Alexander Kanavin wrote: > On Wed, 11 Feb 2026 at 15:41, Rob Woolley <rob.woolleywr@gmail.com> wrote: > >> Are we okay with pinning the temporary staging directory inside the git repository? If so, do you have a preference as to what that directory is called and where it is located? > > I am okay and I have no particular preference. As long as it's in .gitignore. > >> The setuptools_scm uses git tags rather than branches. We may want to consider updating our tagging strategy if we want to use it to automatically describe the commit for packages built on master. One option we have is to set tag_regex in pyproject.toml to a custom pattern. Thus we could create tags like bitbake-v2.16.0 or bitbake-setup-v2.16.0 to provide a reference point for results like bitbake-setup-2.16.1.dev0+gd2e159d85.d20260211 or something simpler like bitbake-setup-2.16.2.dev1 (see https://setuptools-scm.readthedocs.io/en/latest/extending/#version-number-construction) > > The tags available in master branch can be see with 'git tag --merged > master'. It seems that the two most suitable sets are at the end: > > 2024-04-scarthgap > 2024-10-styhead > 2025-04-walnascar > 2025-04.1-walnascar > 2025-10-whinlatter > ... > yocto-5.0 > yocto-5.1 > yocto-5.2 > yocto-5.2.1 > yocto-5.3 > > So we could simply use one of these versions (they match exact release > points), and then add a .relN+hash if that can be done automatically. > And then there's no need to introduce new tag schemes or extra work > for RP. I like the one with the year and release name a bit more, > because it's more informative to the users. > Read https://packaging.python.org/en/latest/specifications/version-specifiers/ and https://peps.python.org/pep-0440/. Specifically: """ Local version identifiers SHOULD NOT be used when publishing upstream projects to a public index server """ The local version is whatever's after a +. So this isn't an option. Also, no yocto- or -<yocto-release> string possible. See https://packaging.python.org/en/latest/specifications/version-specifiers/#public-version-identifiers 2025-04.1 will also not be a valid public version identifier. It needs to be 2025.04.1. So the only viable option would be 5.3 for example (you can strip yocto- automatically in versioningit for example). Now the issue for BitBake repo is that we have the initial release tag on the master branch. This means that the tag is present both on master and 2.16 branch for example. This also means that X.Y.Z.devN may point to two different commits, the N-th one after X.Y.Z release in the release branch, or the N-th one in master. This is of course illegal on PyPI so we'll need to separate them. I'm suggesting to use .devN for master and .postN for release branches. How to make this work with the packaging tool... I don't know. I see that versioningit has customizable hooks so maybe that could be an answer on doing this manually in some way. See https://versioningit.readthedocs.io/en/stable/configuration.html and https://versioningit.readthedocs.io/en/stable/writing-methods.html Cheers, Quentin
diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 00000000..4aff98bb --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,3 @@ +[build-system] +requires = ["setuptools>=61.0.0", "wheel"] +build-backend = "setuptools.build_meta" diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 00000000..3eaa3c49 --- /dev/null +++ b/setup.cfg @@ -0,0 +1,40 @@ +[metadata] +name = bitbake-setup +version = 2.16.0a1 +author = OpenEmbedded BitBake Developers +author_email = bitbake-devel@lists.openembedded.org +description = bitbake-setup +long_description = file: README.md +long_description_content_type = text/markdown +url = https://github.com/openembedded/bitbake.git +license = "GPL-2.0-only and MIT" +classifiers = + Programming Language :: Python :: 3 + License :: OSI Approved :: MIT License + License :: OSI Approved :: GNU General Public License v2 (GPLv2) + Operating System :: POSIX :: Linux + +[options] +package_dir = + = lib +packages = find: +include_package_data = True +zip_safe = False +python_requires = >=3.6 +py_modules = codegen + +install_requires = + ; List any non-standard dependencies BitBake needs + ; BitBake is mostly self-contained, but check its documentation + +[options.packages.find] +where = lib +include = + bb* + bs4* + ply + bitbake_setup + +[options.entry_points] +console_scripts = + bitbake-setup = bitbake_setup.__main__:main
This enables creating a pip package for PyPI with: python3 -m build Signed-off-by: Rob Woolley <rob.woolley@windriver.com> --- pyproject.toml | 3 +++ setup.cfg | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) create mode 100644 pyproject.toml create mode 100644 setup.cfg