| Message ID | 20250716152249.96126-3-richard.purdie@linuxfoundation.org |
|---|---|
| State | Accepted, archived |
| Commit | 945095602e40d54efb8de494218f4a2b25c9969f |
| Headers | show |
| Series | [1/4] asyncrpc: Avoid file not found traceback in logs | expand |
On Wed, Jul 16, 2025 at 9:22 AM Richard Purdie via lists.openembedded.org <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > > It is possible for multple bitbake threads to empty ui_queue in parallel > leading to duplicate console messages and much confusion when debuging. > > Use the lock to extract the queue data which means only one thread will > processing, removing the duplicate out of order messages. > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > lib/bb/event.py | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/lib/bb/event.py b/lib/bb/event.py > index b29f0a55687..2256fed7648 100644 > --- a/lib/bb/event.py > +++ b/lib/bb/event.py > @@ -236,9 +236,12 @@ def fire(event, d): > # If messages have been queued up, clear the queue > global _uiready, ui_queue > if _uiready and ui_queue: > - for queue_event in ui_queue: > + with bb.utils.lock_timeout_nocheck(_thread_lock): > + queue = ui_queue > + ui_queue = [] > + for queue_event in queue: Do we even need this code anymore? fire_ui_handlers() has a loop to process all the events anyway (although, it looks like it probably should have some locking also, maybe: while True: try: i = ui_queue.pop() except IndexError: break fire_ui_handler(i) would work in a lock-less way (or rather, use the GIL as the lock instead). > fire_ui_handlers(queue_event, d) > - ui_queue = [] > + > fire_ui_handlers(event, d) > > def fire_from_worker(event, d): > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#17781): https://lists.openembedded.org/g/bitbake-devel/message/17781 > Mute This Topic: https://lists.openembedded.org/mt/114186788/3616693 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [JPEWhacker@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Thu, 2025-07-17 at 09:29 -0600, Joshua Watt wrote: > On Wed, Jul 16, 2025 at 9:22 AM Richard Purdie via > lists.openembedded.org > <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > > > > It is possible for multple bitbake threads to empty ui_queue in parallel > > leading to duplicate console messages and much confusion when debuging. > > > > Use the lock to extract the queue data which means only one thread will > > processing, removing the duplicate out of order messages. > > > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > > --- > > lib/bb/event.py | 7 +++++-- > > 1 file changed, 5 insertions(+), 2 deletions(-) > > > > diff --git a/lib/bb/event.py b/lib/bb/event.py > > index b29f0a55687..2256fed7648 100644 > > --- a/lib/bb/event.py > > +++ b/lib/bb/event.py > > @@ -236,9 +236,12 @@ def fire(event, d): > > # If messages have been queued up, clear the queue > > global _uiready, ui_queue > > if _uiready and ui_queue: > > - for queue_event in ui_queue: > > + with bb.utils.lock_timeout_nocheck(_thread_lock): > > + queue = ui_queue > > + ui_queue = [] > > + for queue_event in queue: > > Do we even need this code anymore? fire_ui_handlers() has a loop to > process all the events anyway (although, it looks like it probably > should have some locking also, maybe: > > while True: > try: > i = ui_queue.pop() > except IndexError: > break > fire_ui_handler(i) > > > would work in a lock-less way (or rather, use the GIL as the lock instead). We do need this to queue things before the UI handlers are present. There is actually locking present elsewhere in the fire code, it just works on a per event basis rather than for a group, hence the code difference. Cheers, Richard
diff --git a/lib/bb/event.py b/lib/bb/event.py index b29f0a55687..2256fed7648 100644 --- a/lib/bb/event.py +++ b/lib/bb/event.py @@ -236,9 +236,12 @@ def fire(event, d): # If messages have been queued up, clear the queue global _uiready, ui_queue if _uiready and ui_queue: - for queue_event in ui_queue: + with bb.utils.lock_timeout_nocheck(_thread_lock): + queue = ui_queue + ui_queue = [] + for queue_event in queue: fire_ui_handlers(queue_event, d) - ui_queue = [] + fire_ui_handlers(event, d) def fire_from_worker(event, d):
It is possible for multple bitbake threads to empty ui_queue in parallel leading to duplicate console messages and much confusion when debuging. Use the lock to extract the queue data which means only one thread will processing, removing the duplicate out of order messages. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- lib/bb/event.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)