| Message ID | 20230407030715.4394-1-Qi.Chen@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | [bitbake-devel,1/2] runqueue: fix PSI check calculation | expand |
On 2023-04-06 23:07, Chen Qi via lists.openembedded.org wrote: > The current PSI check calculation does not take into consideration > the possibility of the time interval between last check and current > check being much larger than 1s. In fact, the current behavior does > not match what the manual says about BB_PRESSURE_MAX_XXX, even if > the value is set to upper limit, 1000000, we still get many blocks > on new task launch. The difference between 'total' should be divided > by the time interval if it's larger than 1s. Yes! I had a patch to do this but wanted to write a test case using stress . Anyway, it's clearly better and won't allow the occasional new job to slip in when the time diff is small and the pressure hasn't been recalculated by the kernel. It will mean that pressure regulated builds may be a bit slower but I doubt we'll even be able to measure that. Thanks Qi, ../Randy > > Signed-off-by: Chen Qi<Qi.Chen@windriver.com> > --- > bitbake/lib/bb/runqueue.py | 13 +++++++++---- > 1 file changed, 9 insertions(+), 4 deletions(-) > > diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py > index e629ab7e7b..02f1474540 100644 > --- a/bitbake/lib/bb/runqueue.py > +++ b/bitbake/lib/bb/runqueue.py > @@ -198,15 +198,20 @@ class RunQueueScheduler(object): > curr_cpu_pressure = cpu_pressure_fds.readline().split()[4].split("=")[1] > curr_io_pressure = io_pressure_fds.readline().split()[4].split("=")[1] > curr_memory_pressure = memory_pressure_fds.readline().split()[4].split("=")[1] > - exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) > self.rq.max_cpu_pressure > - exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) > self.rq.max_io_pressure > - exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) > self.rq.max_memory_pressure > now = time.time() > - if now - self.prev_pressure_time > 1.0: > + tdiff = now - self.prev_pressure_time > + if tdiff > 1.0: > + exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) / tdiff > self.rq.max_cpu_pressure > + exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) / tdiff > self.rq.max_io_pressure > + exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) / tdiff > self.rq.max_memory_pressure > self.prev_cpu_pressure = curr_cpu_pressure > self.prev_io_pressure = curr_io_pressure > self.prev_memory_pressure = curr_memory_pressure > self.prev_pressure_time = now > + else: > + exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) > self.rq.max_cpu_pressure > + exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) > self.rq.max_io_pressure > + exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) > self.rq.max_memory_pressure > return (exceeds_cpu_pressure or exceeds_io_pressure or exceeds_memory_pressure) > return False > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#14682):https://lists.openembedded.org/g/bitbake-devel/message/14682 > Mute This Topic:https://lists.openembedded.org/mt/98118922/3616765 > Group Owner:bitbake-devel+owner@lists.openembedded.org > Unsubscribe:https://lists.openembedded.org/g/bitbake-devel/unsub [randy.macleod@windriver.com] > -=-=-=-=-=-=-=-=-=-=-=- >
> On Apr 7, 2023, at 11:02 AM, Randy MacLeod <Randy.MacLeod@windriver.com> wrote: > > On 2023-04-06 23:07, Chen Qi via lists.openembedded.org <http://lists.openembedded.org/> wrote: >> The current PSI check calculation does not take into consideration >> the possibility of the time interval between last check and current >> check being much larger than 1s. In fact, the current behavior does >> not match what the manual says about BB_PRESSURE_MAX_XXX, even if >> the value is set to upper limit, 1000000, we still get many blocks >> on new task launch. The difference between 'total' should be divided >> by the time interval if it's larger than 1s. Some ideas about this: The original 1-second interval is to prevent the time interval between two pressure checks too large, but now we can detect longer time intervals and act accordingly, we probably can do the same thing for short time intervals. This may also reduce the build time a bit while having similar CPU pressure - a lot of half a second can stack up significantly. It might be worth testing both solutions. Zheng > > Yes! > I had a patch to do this but wanted to write a test case using stress . > > Anyway, it's clearly better and won't allow the occasional new job to > slip in when the time diff is small and the pressure hasn't been recalculated > by the kernel. It will mean that pressure regulated builds may be a bit slower > but I doubt we'll even be able to measure that. > > Thanks Qi, > > ../Randy > > > >> Signed-off-by: Chen Qi <Qi.Chen@windriver.com> <mailto:Qi.Chen@windriver.com> >> --- >> bitbake/lib/bb/runqueue.py | 13 +++++++++---- >> 1 file changed, 9 insertions(+), 4 deletions(-) >> >> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py >> index e629ab7e7b..02f1474540 100644 >> --- a/bitbake/lib/bb/runqueue.py >> +++ b/bitbake/lib/bb/runqueue.py >> @@ -198,15 +198,20 @@ class RunQueueScheduler(object): >> curr_cpu_pressure = cpu_pressure_fds.readline().split()[4].split("=")[1] >> curr_io_pressure = io_pressure_fds.readline().split()[4].split("=")[1] >> curr_memory_pressure = memory_pressure_fds.readline().split()[4].split("=")[1] >> - exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) > self.rq.max_cpu_pressure >> - exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) > self.rq.max_io_pressure >> - exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) > self.rq.max_memory_pressure >> now = time.time() >> - if now - self.prev_pressure_time > 1.0: >> + tdiff = now - self.prev_pressure_time >> + if tdiff > 1.0: >> + exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) / tdiff > self.rq.max_cpu_pressure >> + exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) / tdiff > self.rq.max_io_pressure >> + exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) / tdiff > self.rq.max_memory_pressure >> self.prev_cpu_pressure = curr_cpu_pressure >> self.prev_io_pressure = curr_io_pressure >> self.prev_memory_pressure = curr_memory_pressure >> self.prev_pressure_time = now >> + else: >> + exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) > self.rq.max_cpu_pressure >> + exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) > self.rq.max_io_pressure >> + exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) > self.rq.max_memory_pressure >> return (exceeds_cpu_pressure or exceeds_io_pressure or exceeds_memory_pressure) >> return False >> >> >> > > -- > # Randy MacLeod > # Wind River Linux > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#14684): https://lists.openembedded.org/g/bitbake-devel/message/14684 > Mute This Topic: https://lists.openembedded.org/mt/98118922/7355053 > Group Owner: bitbake-devel+owner@lists.openembedded.org <mailto:bitbake-devel+owner@lists.openembedded.org> > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [contrib@zhengqiu.net <mailto:contrib@zhengqiu.net>] > -=-=-=-=-=-=-=-=-=-=-=-
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py index e629ab7e7b..02f1474540 100644 --- a/bitbake/lib/bb/runqueue.py +++ b/bitbake/lib/bb/runqueue.py @@ -198,15 +198,20 @@ class RunQueueScheduler(object): curr_cpu_pressure = cpu_pressure_fds.readline().split()[4].split("=")[1] curr_io_pressure = io_pressure_fds.readline().split()[4].split("=")[1] curr_memory_pressure = memory_pressure_fds.readline().split()[4].split("=")[1] - exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) > self.rq.max_cpu_pressure - exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) > self.rq.max_io_pressure - exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) > self.rq.max_memory_pressure now = time.time() - if now - self.prev_pressure_time > 1.0: + tdiff = now - self.prev_pressure_time + if tdiff > 1.0: + exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) / tdiff > self.rq.max_cpu_pressure + exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) / tdiff > self.rq.max_io_pressure + exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) / tdiff > self.rq.max_memory_pressure self.prev_cpu_pressure = curr_cpu_pressure self.prev_io_pressure = curr_io_pressure self.prev_memory_pressure = curr_memory_pressure self.prev_pressure_time = now + else: + exceeds_cpu_pressure = self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) > self.rq.max_cpu_pressure + exceeds_io_pressure = self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) > self.rq.max_io_pressure + exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) > self.rq.max_memory_pressure return (exceeds_cpu_pressure or exceeds_io_pressure or exceeds_memory_pressure) return False
The current PSI check calculation does not take into consideration the possibility of the time interval between last check and current check being much larger than 1s. In fact, the current behavior does not match what the manual says about BB_PRESSURE_MAX_XXX, even if the value is set to upper limit, 1000000, we still get many blocks on new task launch. The difference between 'total' should be divided by the time interval if it's larger than 1s. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> --- bitbake/lib/bb/runqueue.py | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)