[RFC,yocto-autobuilder2] builders: Merge build request priority support from upstream buildbot

From: Yoann Congal <yoann.congal@smile.fr>

We use a custom prioritisation function to assigning build requests to
workers.

Buildbot gained the feature of specifying the priority of a
build in v3.9.0[0] (our custom function predates that so ignore the
field entirely).

Merge the priority support added to the buildbot default sort function
into our custom function (see [0]).

The resulting order should be:
* First priority (higher priority goes first, defaults to 0)
* Then, time (oldest build goes first with bonuses for certain builds)
* Then, builder name (last resort)

With this patch, the priority field of the buildbot UI when starting a
build can be used to specify a less important build that can goes after
normal builds (e.g. set priority to -1 for hunting an AB-INT issue).

[0]: https://github.com/buildbot/buildbot/commit/bf1cd1321067fd92408fe6fc18d00d2edb5f2770

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
---
RFC:
* I could not test any of this...
* I don't know if that is a real problem to solve. How often the AB
  cluster has pending requests that can't be satisfied for the lack of
  available workers?
---
 builders.py | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

Le ven. 9 janv. 2026 à 23:50, Yoann Congal <yoann.congal@smile.fr> a écrit :

> From: Yoann Congal <yoann.congal@smile.fr>
>
> We use a custom prioritisation function to assigning build requests to
> workers.
>
> Buildbot gained the feature of specifying the priority of a
> build in v3.9.0[0] (our custom function predates that so ignore the
> field entirely).
>
> Merge the priority support added to the buildbot default sort function
> into our custom function (see [0]).
>
> The resulting order should be:
> * First priority (higher priority goes first, defaults to 0)
> * Then, time (oldest build goes first with bonuses for certain builds)
> * Then, builder name (last resort)
>
> With this patch, the priority field of the buildbot UI when starting a
> build can be used to specify a less important build that can goes after
> normal builds (e.g. set priority to -1 for hunting an AB-INT issue).
>
> [0]:
> https://github.com/buildbot/buildbot/commit/bf1cd1321067fd92408fe6fc18d00d2edb5f2770
>
> Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
> ---
> RFC:
> * I could not test any of this...
> * I don't know if that is a real problem to solve. How often the AB
>   cluster has pending requests that can't be satisfied for the lack of
>   available workers?
>

To answer my own question: It happens.
As of now (2026-01-14 15:45:35+01:00) every worker has multiple builds and
the buildrequest queue has 6 pending requests (oldest is 1 hour old).
So, maybe this patch might be useful.


> ---
>  builders.py | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/builders.py b/builders.py
> index b7379ec..3d55e15 100644
> --- a/builders.py
> +++ b/builders.py
> @@ -244,6 +244,12 @@ def prioritizeBuilders(master, builders):
>
>      @defer.inlineCallbacks
>      def transform(bldr):
> +        # Sort primarily highest priority of build requests
> +        priority = yield bldr.get_highest_priority()
> +        if priority is None:
> +            # for builders that do not have pending buildrequest, we just
> use large number
> +            priority = -math.inf
> +        # Break ties using the time of oldest build request (with bonus)
>          time = yield bldr.getOldestRequestTime()
>          #log.msg("Considering %s" % bldr.name)
>          if time is None:
> @@ -254,7 +260,7 @@ def prioritizeBuilders(master, builders):
>                  time = time - builder_bonuses[bldr.name]
>                  #log.msg("Adding %s so %s" % (str(builder_bonuses[
> bldr.name]), str(time)))
>
> -        return (time, bldr)
> +        return (-priority, time, bldr)
>
>      transformed = yield defer.gatherResults(
>          [transform(bldr) for bldr in builders])
> @@ -262,13 +268,13 @@ def prioritizeBuilders(master, builders):
>      # sort the transformed list synchronously, comparing None to the end
> of
>      # the list
>      def transformedKey(a):
> -        (date, builder) = a
> -        return (date, builder.name)
> +        (priority, date, builder) = a
> +        return (priority, date, builder.name)
>
>      transformed.sort(key=transformedKey)
>
>      # and reverse the transform
> -    rv = [xf[1] for xf in transformed]
> +    rv = [xf[2] for xf in transformed]
>      #log.msg("Using %s" % str(rv))
>      return rv
>
>