mbox series

[v2,0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

Message ID 20250122232411.133-1-egyszeregy@freemail.hu
Headers show
Series bitbake: knotty/progressbar: Optimize footer update and prints. | expand

Message

Livius Jan. 22, 2025, 11:24 p.m. UTC
s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=845; bh=wCAz1kJFGU6x5XIcVruNfIrdFNZ+cbFWs4l3pbMJK/0=;

	b=F+Iqee6Wp0hEaYJRrNW03FBQHF+WSypPxCENwTDK/4JtMAekPNAUvho6F9wm7hHE

	s4H4t1w7oXaNXyJvJzlzTLx3HtTx5aZIpFrnM49AIEf/iwFffEnEUn3V2+Cm0RskbbM

	MLChdAN5QbJXRgWakwIVRcNkMhKRJP8Tc3IQVqGr/XWqCtfzkzKyiWB5ulqCkKaUBtK

	S9z/DSJZQWtnyuJbx8b40CSajkmkmQx707KIPH2FJKHgvS1uNSck7EzErsrTWFLIi8n

	GL13wFt3XE5avGIqKfxrLF/0RgNEyjk3Gh2B0K/W4b2uY7xI4ewtiiz9AfXsdI8hlqF

	A8zJwDXzPw==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

- Check file descriptor in ProgressBar() class init to see if it is resiz=
able.
- Optimize printing in footer update with use a StringIO buffer.
- Use 10 Hz refresh rate (FPS) for footer update.
- Cleaning and refactoring prints in code, print() functions were elimina=
ted from all loops for better performance.

Benjamin Sz=C5=91ke (4):
  bitbake: progressbar: Check resizable file descriptor.
  bitbake: knotty: Use a StringIO buffer for update footer.
  bitbake: knotty: Use 10 Hz refresh rate (FPS) for footer update.
  bitbake: knotty: print() was eliminated from all loops for better
    performance.

 lib/bb/ui/knotty.py            | 69 ++++++++++++++++++++++++----------
 lib/progressbar/progressbar.py | 25 +++++++-----
 2 files changed, 66 insertions(+), 28 deletions(-)

--=20
2.47.1.windows.2

Comments

Richard Purdie Jan. 23, 2025, 12:01 a.m. UTC | #1
On Thu, 2025-01-23 at 00:24 +0100, Livius via lists.openembedded.org
wrote:
> From: Benjamin Szőke <egyszeregy@freemail.hu>
> 
> - Check file descriptor in ProgressBar() class init to see if it is
> resizable.
> - Optimize printing in footer update with use a StringIO buffer.
> - Use 10 Hz refresh rate (FPS) for footer update.
> - Cleaning and refactoring prints in code, print() functions were
> eliminated from all loops for better performance.

This series looks better, thanks. I still have concerns about the first
patch and I've replied to that one.

One question that still isn't answered is how you're measuring this
performance improvement. What issues were you seeing (apart from the
frequent console updates)?

Cheers,

Richard
Livius Jan. 25, 2025, 11:35 a.m. UTC | #2
Performance improvement is quite trivial that will happen. Not allowed to write to stdout or any unbuffered file while the code is running inside a loop in any script language or programming language.

I started to look for the performance issues because my Yocto projects started to build slowly from scarthgap release in any physical machines (only in a virtual machine it can run fast yet).
https://lists.openembedded.org/g/bitbake-devel/topic/109854844

Yesterday, I found the bug which releated to bitbake Event logger (workaround to disable it). Soon I will send a Patch v3 for footer update, but not this was the root cause, i will also send a patch for fix even logger class logging which has the real critical problem.