diff mbox series

[yocto-autobuilder2,RFC] master.cfg: Replace pkg_resources with importlib.metadata

Message ID 20260417013758.91120-2-tim.orling@konsulko.com
State New
Headers show
Series [yocto-autobuilder2,RFC] master.cfg: Replace pkg_resources with importlib.metadata | expand

Commit Message

Tim Orling April 17, 2026, 1:37 a.m. UTC
pkg_resources (from setuptools) is deprecated. Replace the fake
Distribution/EntryPoint/working_set manipulation with
importlib.metadata equivalents:

- Intercept importlib.metadata.version() for the local plugin, since
  buildbot's Application() calls it at import time before the plugin
  is registered.
- Use importlib.metadata.EntryPoint directly instead of the old
  pkg_resources.EntryPoint.parse() plus fake Distribution approach.
- Remove the now-unused os import.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
This has been working for me on my local AB cluster, where the
controller is running on a Debian 13 AMD Ryzen system.

 master.cfg | 61 +++++++++++++++++++++++++++---------------------------
 1 file changed, 31 insertions(+), 30 deletions(-)

Comments

Richard Purdie April 17, 2026, 9 a.m. UTC | #1
On Thu, 2026-04-16 at 18:37 -0700, Tim Orling via lists.yoctoproject.org wrote:
> pkg_resources (from setuptools) is deprecated. Replace the fake
> Distribution/EntryPoint/working_set manipulation with
> importlib.metadata equivalents:
> 
> - Intercept importlib.metadata.version() for the local plugin, since
>   buildbot's Application() calls it at import time before the plugin
>   is registered.
> - Use importlib.metadata.EntryPoint directly instead of the old
>   pkg_resources.EntryPoint.parse() plus fake Distribution approach.
> - Remove the now-unused os import.
> 
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> ---
> This has been working for me on my local AB cluster, where the
> controller is running on a Debian 13 AMD Ryzen system.

Nice!

I think Michael gave up on the local install and is building/running
things a bit differently. I did find this approach handy for local
development and did try and get it working again but failed and
couldn't spend more time on it.

I'm happy to merge this and replace what is there as it has to be an
improvement, thanks!

Cheers,

Richard
diff mbox series

Patch

diff --git a/master.cfg b/master.cfg
index 63119ca..55312ef 100644
--- a/master.cfg
+++ b/master.cfg
@@ -1,46 +1,47 @@ 
 # -*- python -*-
 # ex: set filetype=python:
 
-import os
 import importlib
-import pkg_resources
-
-from buildbot.plugins import *
-from buildbot.plugins import db
-from yoctoabb import builders, config, schedulers, workers, services, www
+import importlib.metadata
+from importlib.metadata import EntryPoint
 
 #
-# Prepare to feel ill
+# Buildbot discovers www plugins via entry points. Since the yocto_console_view
+# plugin lives in this repo rather than being installed as a package, we need
+# two workarounds:
 #
-# Buildbot uses plugins from pkg_resources, i.e. things which are installed
-# For the Yocto autobuilder, we want the plugin from this repo/directory.
+# 1. Buildbot's Application() calls importlib.metadata.version() at import
+#    time, so we intercept that to return a version for our local plugin
+#    before any yoctoabb imports occur.
 #
-# Firstly we therefore have to create a dummy pkg_resources entry/distribution
-# which we add into the global working set. Ugly. It gets worse.
+# 2. The get_plugins('www') call from www/service.py happens before this
+#    master.cfg file is parsed, so we create an EntryPoint directly and
+#    register it into buildbot's internal plugin DB.
 #
-# The get_plugins('www') call from www/service.py happens before this
-# master.cfg file is parsed, which means our plugin won't be found. There
-# is no API to rescan for plugins. We therefore so some horrible internal
-# API monkey patching to add ourselves into the internal list.
 
-# Create a fake distribution to insert into the global working_set
-us = os.path.dirname(os.path.realpath(__file__))
-d = pkg_resources.Distribution(us + "/yocto_console_view", metadata=None, project_name="yocto_console_view", version="0.0.1")
+# Handle importlib.metadata.version() for our uninstalled local plugin.
+# This must happen before importing yoctoabb, since that triggers
+# Application().__init__ which calls importlib.metadata.version().
+_orig_metadata_version = importlib.metadata.version
+def _local_metadata_version(name):
+    if name == 'yoctoabb.yocto_console_view.yocto_console_view':
+        return '0.0.1'
+    return _orig_metadata_version(name)
+importlib.metadata.version = _local_metadata_version
 
-# Create the fake entry point definition
-ep = pkg_resources.EntryPoint.parse("yocto_console_view = yoctoabb.yocto_console_view.yocto_console_view:ep", dist=d)
-
-# Add the mapping to the fake EntryPoint
-d._ep_map = {'buildbot.www': {'yocto_console_view': ep}}
+from buildbot.plugins import *
+from buildbot.plugins import db
+from yoctoabb import builders, config, schedulers, workers, services, www
 
-# Add the fake distribution to the global working_set
-pkg_resources.working_set.add(d)
+ep_entry = EntryPoint(
+    name='yocto_console_view',
+    value='yoctoabb.yocto_console_view.yocto_console_view:ep',
+    group='buildbot.www'
+)
 
-# Now monkey patch us into buildbot's plugins DB
-for entry in pkg_resources.iter_entry_points('buildbot.www'):
-    if entry.name == "yocto_console_view":
-        plugindb = db._DB._namespaces['www']
-        plugindb._real_tree.add(entry.name, db._PluginEntry('buildbot.www', entry, plugindb._load_entry))
+# Register the plugin directly into buildbot's plugin DB
+plugindb = db._DB._namespaces['www']
+plugindb._real_tree.add(ep_entry.name, db._PluginEntry('buildbot.www', ep_entry, plugindb._load_entry))
 
 
 # supports re-loading configuration with buildbot sighup, meaning we can change