[1/2] runqueue: decrease the verbosity of pressure messages

By default all the pressure changes are printed on the console and
generate a lot of spam to users using the BB_PRESSURE_* variables.
Decrease the default verbosity to bb.debug (level 1, so can be seen with
-D).

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 lib/bb/runqueue.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

On Fri, Apr 11, 2025 at 4:46 PM Antonin Godard via
lists.yoctoproject.org
<antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
>
> By default all the pressure changes are printed on the console and
> generate a lot of spam to users using the BB_PRESSURE_* variables.
> Decrease the default verbosity to bb.debug (level 1, so can be seen with
> -D).

Isn't "a lot of these messages" a proof that the BB_PRESSURE_*
variables aren't set correctly?

I find them quite useful and using -D by default in my builds would
bring even "more spam". But in worse case I'll just revert this for my
builds.

Cheers,

> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
>  lib/bb/runqueue.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
> index 8fadc8338..adfe3c7a1 100644
> --- a/lib/bb/runqueue.py
> +++ b/lib/bb/runqueue.py
> @@ -219,7 +219,7 @@ class RunQueueScheduler(object):
>              pressure_state = (exceeds_cpu_pressure, exceeds_io_pressure, exceeds_memory_pressure)
>              pressure_values = (round(cpu_pressure,1), self.rq.max_cpu_pressure, round(io_pressure,1), self.rq.max_io_pressure, round(memory_pressure,1), self.rq.max_memory_pressure)
>              if hasattr(self, "pressure_state") and pressure_state != self.pressure_state:
> -                bb.note("Pressure status changed to CPU: %s, IO: %s, Mem: %s (CPU: %s/%s, IO: %s/%s, Mem: %s/%s) - using %s/%s bitbake threads" % (pressure_state + pressure_values + (len(self.rq.runq_running.difference(self.rq.runq_complete)), self.rq.number_tasks)))
> +                bb.debug(1, "Pressure status changed to CPU: %s, IO: %s, Mem: %s (CPU: %s/%s, IO: %s/%s, Mem: %s/%s) - using %s/%s bitbake threads" % (pressure_state + pressure_values + (len(self.rq.runq_running.difference(self.rq.runq_complete)), self.rq.number_tasks)))
>              self.pressure_state = pressure_state
>              return (exceeds_cpu_pressure or exceeds_io_pressure or exceeds_memory_pressure)
>          elif self.rq.max_loadfactor:
>
> --
> 2.47.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#6736): https://lists.yoctoproject.org/g/docs/message/6736
> Mute This Topic: https://lists.yoctoproject.org/mt/112211584/3617156
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [martin.jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Hi Martin,

On Fri Apr 11, 2025 at 9:09 PM CEST, Martin Jansa wrote:
> On Fri, Apr 11, 2025 at 4:46 PM Antonin Godard via
> lists.yoctoproject.org
> <antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
>>
>> By default all the pressure changes are printed on the console and
>> generate a lot of spam to users using the BB_PRESSURE_* variables.
>> Decrease the default verbosity to bb.debug (level 1, so can be seen with
>> -D).
>
> Isn't "a lot of these messages" a proof that the BB_PRESSURE_*
> variables aren't set correctly?
>
> I find them quite useful and using -D by default in my builds would
> bring even "more spam". But in worse case I'll just revert this for my
> builds.

I see how these messages can be useful.

To my understanding, having a low value for BB_PRESSURE_* variables will
generate a lot of these messages. To me, it does not necessarily mean that the
value isn't properly set, it just means that you want to limit the pressure a
lot.

Maybe the default behavior of printing everything is good for adjustments, I
just think it'd be convenient for user to have a way to disable those once they
are happy with their pressure configuration.

I thought first of a BB_PRESSURE_VERBOSE or BB_PRESSURE_QUIET variable. It
would just allow enabling/disabling these messages. But on second thought, I'm
not sure an extra variable is necessary when we could just move these messages
to debug messages. Other variables that limit the load do not generate such
"spam" (take BB_LOADFACTOR_MAX for example).

What do you think?

Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

On Tue, Apr 15, 2025 at 1:52 PM Antonin Godard
<antonin.godard@bootlin.com> wrote:
>
> Hi Martin,
>
> On Fri Apr 11, 2025 at 9:09 PM CEST, Martin Jansa wrote:
> > On Fri, Apr 11, 2025 at 4:46 PM Antonin Godard via
> > lists.yoctoproject.org
> > <antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
> >>
> >> By default all the pressure changes are printed on the console and
> >> generate a lot of spam to users using the BB_PRESSURE_* variables.
> >> Decrease the default verbosity to bb.debug (level 1, so can be seen with
> >> -D).
> >
> > Isn't "a lot of these messages" a proof that the BB_PRESSURE_*
> > variables aren't set correctly?
> >
> > I find them quite useful and using -D by default in my builds would
> > bring even "more spam". But in worse case I'll just revert this for my
> > builds.
>
> I see how these messages can be useful.
>
> To my understanding, having a low value for BB_PRESSURE_* variables will
> generate a lot of these messages. To me, it does not necessarily mean that the
> value isn't properly set, it just means that you want to limit the pressure a
> lot.
>
> Maybe the default behavior of printing everything is good for adjustments, I
> just think it'd be convenient for user to have a way to disable those once they
> are happy with their pressure configuration.
>
> I thought first of a BB_PRESSURE_VERBOSE or BB_PRESSURE_QUIET variable. It
> would just allow enabling/disabling these messages. But on second thought, I'm
> not sure an extra variable is necessary when we could just move these messages
> to debug messages. Other variables that limit the load do not generate such
> "spam" (take BB_LOADFACTOR_MAX for example).

I guess you're using the default knotty where I agree that these
messages from pressure are more annoying, because there isn't much
else printed by default.

Most of my builds are on jenkins or run with tee, so I get a lot of
messages about e.g. starting and stopping tasks and almost never
notice the few pressure messages between them (unless it's IMHO badly
configured and changes regulation after almost every task).

> What do you think?

Do what you want (and others agree with) and I'll deal with it for my
builds if needed (I might just disable pressure regulation as I don't
find it that useful as it cannot prevent OOMK anyway and the task
granularity doesn't prevent it from triggering multiple heavy and log
do_compile tasks at the same time while the pressure was still low and
it gets overloaded shortly after - still hoping that
https://github.com/ninja-build/ninja/pull/2506 will be merged one
day).