diff mbox series

[1.50,5/8] cooker: Handle parse threads disappearing to avoid hangs

Message ID fc4a8dbed4174e07534666860466173a171aae73.1637547674.git.anuj.mittal@intel.com
State Accepted, archived
Commit 6e746ccf8977a78f1565c6b2fe65aaede260e1bb
Headers show
Series [1.50,1/8] bitbake: correct the collections vs collections.abc deprecation | expand

Commit Message

Mittal, Anuj Nov. 22, 2021, 2:24 a.m. UTC 
From: Richard Purdie <richard.purdie@linuxfoundation.org>

If one of the parse threads disappears during parsing for some reason, bitbake
currently hangs. Avoid this (and zombie threads hanging around) by joining()
threads which have exited.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dc86a533d951d13643ce446533370da804782afc)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
 lib/bb/cooker.py | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
diff mbox series

Patch

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index b041d2a0..f12f4caa 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -2036,6 +2036,7 @@  class Parser(multiprocessing.Process):
                 result = pending.pop()
             else:
                 try:
+                    time.sleep(0.25)
                     job = self.jobs.pop()
                 except IndexError:
                     self.results.close()
@@ -2214,7 +2215,7 @@  class CookerParser(object):
             yield not cached, mc, infos
 
     def parse_generator(self):
-        while True:
+        while self.processes:
             if self.parsed >= self.toparse:
                 break
 
@@ -2228,6 +2229,14 @@  class CookerParser(object):
                     raise value
                 else:
                     yield result
+            for process in self.processes.copy():
+                if not process.is_alive():
+                    process.join()
+                    self.processes.remove(process)
+
+        if not (self.parsed >= self.toparse):
+            raise bb.parse.ParseError("Not all recipes parsed, parser thread killed/died? Exiting.", None)
+
 
     def parse_next(self):
         result = []