[yocto-autobuilder2] schedulers: metrics: Start again with master for the daily full download

Change the order of metrics branches:
* First, master at 5am which may do the full CVE download (might take
  1 hour to finish).
* Then, the other branches starting at 6am to allow them to use the
  fresh download.
* And, finally, generate global charts data from individual branch
  metrics (at 8am)

Compared with commit 839c3cedbf6f ("schedulers: metrics: Start with master for the daily full download"),
the last part is new.

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
---
 schedulers.py | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

(Replying to myself to keep conclusion of an IRC discussion)

Le sam. 16 mars 2024 à 00:29, Yoann Congal <yoann.congal@smile.fr> a écrit :

> Change the order of metrics branches:
> * First, master at 5am which may do the full CVE download (might take
>   1 hour to finish).
> * Then, the other branches starting at 6am to allow them to use the
>   fresh download.
> * And, finally, generate global charts data from individual branch
>   metrics (at 8am)
>

That would not work since the master metrics can go well above 1h (e.g.
https://autobuilder.yoctoproject.org/typhoon/#/builders/138/builds/1343)

What we want is:
* A triggering builder (see
https://docs.buildbot.net/3.8.0/manual/configuration/steps/trigger.html)
that
  * for each branch,
    * trigger a builder that do the run-cvecheck (starting with master for
the NVD database download)
  * Finish with the chart generation (run-cvechart)

That way, we will have proper ordering/dependencies between metrics
computing and chart generation.

NB: Trying to order builders on the AB based on estimated build duration
does not work because workers are quite loaded and the pressure regulation
can extend the build time quite a bit.


> Compared with commit 839c3cedbf6f ("schedulers: metrics: Start with master
> for the daily full download"),
> the last part is new.
>
> Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
> ---
>



>  schedulers.py | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/schedulers.py b/schedulers.py
> index a30383f..67cc739 100644
> --- a/schedulers.py
> +++ b/schedulers.py
> @@ -497,12 +497,17 @@
> schedulers.append(sched.Nightly(name='nightly-meta-oe-mirror',
> branch='master',
>  schedulers.append(sched.Nightly(name='nightly-reproducible-meta-oe',
> branch='master', properties=parent_default_props('reproducible-meta-oe'),
>                    builderNames=['reproducible-meta-oe'], hour=2,
> minute=0, dayOfWeek=5))
>
> -# Run metrics at 3-7am each day
> +# Run metrics at 5-8am each day
> +# Starts with master which may do a full CVE database download and start
> other
> +# branches 1 hour later to allow them to re-use the fresh download.
> +schedulers.append(sched.Nightly(name='nightly-metrics', branch='master',
> properties=parent_default_props('metrics'), builderNames=['metrics'],
> hour=5, minute=0))
>  schedulers.append(sched.Nightly(name='nightly-metrics-dunfell',
> branch='dunfell', properties=parent_default_props('metrics', 'dunfell'),
> builderNames=['metrics'], hour=6, minute=0, codebases = {'' : {'branch' :
> 'dunfell'}}))
>  schedulers.append(sched.Nightly(name='nightly-metrics-kirkstone',
> branch='kirkstone', properties=parent_default_props('metrics',
> 'kirkstone'), builderNames=['metrics'], hour=6, minute=15, codebases = {''
> : {'branch' : 'kirkstone'}}))
>  schedulers.append(sched.Nightly(name='nightly-metrics-mickledore',
> branch='mickledore', properties=parent_default_props('metrics',
> 'mickledore'), builderNames=['metrics'], hour=6, minute=30, codebases = {''
> : {'branch' : 'mickledore'}}))
>  schedulers.append(sched.Nightly(name='nightly-metrics-nanbield',
> branch='nanbield', properties=parent_default_props('metrics', 'nanbield'),
> builderNames=['metrics'], hour=6, minute=45, codebases = {'' : {'branch' :
> 'nanbield'}}))
> -schedulers.append(sched.Nightly(name='nightly-metrics', branch='master',
> properties=parent_default_props('metrics'), builderNames=['metrics'],
> hour=7, minute=0))
> +# Generate charts after every branch had its metrics computed
> +schedulers.append(sched.Nightly(name='nightly-metrics-charts',
> branch='master', properties=parent_default_props('metrics'),
> builderNames=['metrics-charts'], hour=8, minute=0))
> +
>
>  # Run check-layer-nightly twice a week for nanbield
>  schedulers.append(sched.Nightly(name='nightly-check-layer-nanbield',
> properties=parent_default_props('check-layer-nightly', 'nanbield'),
> --
> 2.39.2
>
>