| Message ID | 20260423102950.697501-2-corentin.guillevic@smile.fr |
|---|---|
| State | New |
| Headers | show |
| Series | cooker: limit the execution of the hashequiv client/server ping | expand |
Hi, On Thu Apr 23, 2026 at 12:29 PM CEST, Corentin Guillevic via lists.openembedded.org wrote: > From: Corentin Guillevic <corentin.guillevic@smile.fr> > > The ping method is used to test the connection to the hash equivalence server. > However, if the server is unavailable, the bitbake server has to wait several > minutes for the system to terminate. > > While waiting for a response from the hash server, the bitbake server does not > provide any feedback to the bitbake client. The latter then closes the link and > fails with the error: > > Timeout while waiting for a reply from the bitbake server (60s at 09:50:15.377337) > > Adding a timeout to the ping step allows quicker action when a remote server is > unavailable, and prevents disconnection of the client due to timeout. > The default timeout is 10 seconds, but this can be set using the > BB_HASHSERVER_TIMEOUT variable. Can you please document this new variable in doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst in the same commit? Thanks, Antonin
On Thu Apr 23, 2026 at 12:29 PM CEST, Corentin Guillevic via lists.openembedded.org wrote: > From: Corentin Guillevic <corentin.guillevic@smile.fr> > > The ping method is used to test the connection to the hash equivalence server. > However, if the server is unavailable, the bitbake server has to wait several > minutes for the system to terminate. > > While waiting for a response from the hash server, the bitbake server does not > provide any feedback to the bitbake client. The latter then closes the link and > fails with the error: > > Timeout while waiting for a reply from the bitbake server (60s at 09:50:15.377337) > > Adding a timeout to the ping step allows quicker action when a remote server is > unavailable, and prevents disconnection of the client due to timeout. > The default timeout is 10 seconds, but this can be set using the > BB_HASHSERVER_TIMEOUT variable. > > Signed-off-by: Corentin Guillevic <corentin.guillevic@smile.fr> > --- Hi Corentin, Thanks for your patch. It looks like this is triggering some oe-selftest failures: 2026-04-24 13:28:03,744 - oe-selftest - INFO - prservice.BitbakePrTests.test_import_export_override_db (subunit.RemotedTestCase) 2026-04-24 13:28:03,746 - oe-selftest - INFO - ... FAIL ... File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/prservice.py", line 115, in test_import_export_override_db self.run_test_pr_export_import('m4', replace_current_db=False) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/prservice.py", line 89, in run_test_pr_export_import self.increment_package_pr(package_name) ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/prservice.py", line 61, in increment_package_pr self.assertEqual(res.status, 0, msg=res.output) ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/unittest/case.py", line 907, in assertEqual assertion_func(first, second, msg=msg) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/unittest/case.py", line 900, in _baseAssertEqual raise self.failureException(msg) AssertionError: 1 != 0 : NOTE: Reconnecting to bitbake server... NOTE: Retrying server connection (#1)... (13:28:02.531366) NOTE: Started PRServer with DBfile: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1371670/cache/prserv.sqlite3, Address: 127.0.0.1:37647, PID: 2273277 ERROR: PRservice localhost:37647 not available ERROR: Unable to start PR Server, exiting, check the bitbake-cookerdaemon.log ... 2026-04-24 13:28:06,081 - oe-selftest - INFO - prservice.BitbakePrTests.test_import_export_replace_db (subunit.RemotedTestCase) 2026-04-24 13:28:06,081 - oe-selftest - INFO - ... FAIL ... 2026-04-24 13:28:08,453 - oe-selftest - INFO - prservice.BitbakePrTests.test_pr_service_deb_arch_dep (subunit.RemotedTestCase) 2026-04-24 13:28:08,453 - oe-selftest - INFO - ... FAIL ... 2026-04-24 13:28:10,791 - oe-selftest - INFO - prservice.BitbakePrTests.test_pr_service_deb_arch_indep (subunit.RemotedTestCase) 2026-04-24 13:28:10,792 - oe-selftest - INFO - ... FAIL ... 2026-04-24 13:28:13,383 - oe-selftest - INFO - prservice.BitbakePrTests.test_pr_service_ipk_arch_dep (subunit.RemotedTestCase) 2026-04-24 13:28:13,384 - oe-selftest - INFO - ... FAIL ... 2026-04-24 13:28:15,719 - oe-selftest - INFO - prservice.BitbakePrTests.test_pr_service_ipk_arch_indep (subunit.RemotedTestCase) 2026-04-24 13:28:15,720 - oe-selftest - INFO - ... FAIL ... 2026-04-24 13:28:18,037 - oe-selftest - INFO - prservice.BitbakePrTests.test_pr_service_rpm_arch_dep (subunit.RemotedTestCase) 2026-04-24 13:28:18,038 - oe-selftest - INFO - ... FAIL ... 2026-04-24 13:28:20,344 - oe-selftest - INFO - prservice.BitbakePrTests.test_pr_service_rpm_arch_indep (subunit.RemotedTestCase) 2026-04-24 13:28:20,345 - oe-selftest - INFO - ... FAIL https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3834 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3744 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/3624 Can you have a look at the issue? Thanks, Mathieu
diff --git a/lib/bb/asyncrpc/client.py b/lib/bb/asyncrpc/client.py index 17b72033b..7ef76c90c 100644 --- a/lib/bb/asyncrpc/client.py +++ b/lib/bb/asyncrpc/client.py @@ -199,8 +199,8 @@ class AsyncClient(object): self.check_invoke_error(result) return result - async def ping(self): - return await self.invoke({"ping": {}}) + async def ping(self, timeout): + return await asyncio.wait_for(self.invoke({"ping": {}}), timeout) async def __aenter__(self): return self diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 4b6ba3196..0bc9b7aec 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -342,7 +342,7 @@ according to https://docs.yoctoproject.org/dev-manual/hashequivserver.html""".fo if upstream: try: with hashserv.create_client(upstream) as client: - client.ping() + client.ping(self.data.getVar("BB_HASHSERVER_TIMEOUT") or 10) except ImportError as e: bb.fatal(""""Unable to use hash equivalence server at '%s' due to missing or incorrect python module: %s @@ -352,7 +352,7 @@ Please install the needed module on the build host, or use an environment contai - or set up pip venv You can also remove the BB_HASHSERVE_UPSTREAM setting, but this may result in significantly longer build times as bitbake will be unable to reuse prebuilt sstate artefacts.""" % (upstream, repr(e))) - except ConnectionError as e: + except (ConnectionError, TimeoutError) as e: bb.warn("Unable to connect to hash equivalence server at '%s', please correct or remove BB_HASHSERVE_UPSTREAM:\n%s" % (upstream, repr(e))) upstream = None