lib/bb/cooker.py: make BB_HASHSERVE_UPSTREAM issues fatal, write specific error messages

From: Alexander Kanavin <alex@linutronix.de>

Previously if the server couldn't be contacted or websockets module was not installed
bitbake would only print a warning and continue, resulting in a degraded user
experience due to inability to use the configured sstate server.

Let's consider these as fatal misconfigurations, so that users can address
the issue properly and not wonder why builds are taking forever.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 bitbake/lib/bb/cooker.py | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

On Tue, 2025-02-25 at 13:09 +0100, Alexander Kanavin via lists.openembedded.org wrote:
> From: Alexander Kanavin <alex@linutronix.de>
> 
> Previously if the server couldn't be contacted or websockets module was not installed
> bitbake would only print a warning and continue, resulting in a degraded user
> experience due to inability to use the configured sstate server.
> 
> Let's consider these as fatal misconfigurations, so that users can address
> the issue properly and not wonder why builds are taking forever.
> 
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
>  bitbake/lib/bb/cooker.py | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
> index 5b885cddd7e..668596c5cdb 100644
> --- a/bitbake/lib/bb/cooker.py
> +++ b/bitbake/lib/bb/cooker.py
> @@ -316,10 +316,15 @@ class BBCooker:
>                      try:
>                          with hashserv.create_client(upstream) as client:
>                              client.ping()
> -                    except (ConnectionError, ImportError) as e:
> -                        bb.warn("BB_HASHSERVE_UPSTREAM is not valid, unable to connect hash equivalence server at '%s': %s"
> +                    except ConnectionError as e:
> +                        bb.fatal("Unable to connect to hash equivalence server at '%s', please correct or remove BB_HASHSERVE_UPSTREAM:\n%s"
> +                                 % (upstream, repr(e)))
> +                    except ImportError as e:
> +                        bb.fatal(""""Unable to use hash equivalence server at '%s' due to missing or incorrect python module:
> +%s
> +Please install the needed module on the build host, or use scripts/install-buildtools from openembedded-core.
> +You can also adjust or remove BB_HASHSERVE_UPSTREAM setting, but this may result in significantly longer build times."""
>                                   % (upstream, repr(e)))
> -                        upstream = None
>  
>                  self.hashservaddr = "unix://%s/hashserve.sock" % self.data.getVar("TOPDIR")
>                  self.hashserv = hashserv.create_server(
> 

I agree we should split the messages and I'm ok with making ImportError
fatal but I'm not sure we should change ConnectionError as it means a
single connection failure to an overloaded server will abort the build.
We've seen this problem before which is why it is a warning.

We can't really reference oe-core from bitbake unfortunately either.
We'd probably need to link to docs at that point.

Cheers,

Richard

On Tue, 25 Feb 2025 at 14:46, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:

> I agree we should split the messages and I'm ok with making ImportError
> fatal but I'm not sure we should change ConnectionError as it means a
> single connection failure to an overloaded server will abort the build.
> We've seen this problem before which is why it is a warning.

Sure, I can tweak this.

> We can't really reference oe-core from bitbake unfortunately either.
> We'd probably need to link to docs at that point.

Here I'm not sure how to rephrase. We can't mention oe-core at all? Or
is it okay to say smth like

"Install the missing module on the build host. If you are using
bitbake with oe-core, you can also use buildtools" etc.

If you can give me a canned error message that I can copy paste,
that'd be best :D

Alex

On Tue, 2025-02-25 at 16:51 +0100, Alexander Kanavin wrote:
> On Tue, 25 Feb 2025 at 14:46, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> 
> > I agree we should split the messages and I'm ok with making ImportError
> > fatal but I'm not sure we should change ConnectionError as it means a
> > single connection failure to an overloaded server will abort the build.
> > We've seen this problem before which is why it is a warning.
> 
> Sure, I can tweak this.
> 
> > We can't really reference oe-core from bitbake unfortunately either.
> > We'd probably need to link to docs at that point.
> 
> Here I'm not sure how to rephrase. We can't mention oe-core at all? Or
> is it okay to say smth like
> 
> "Install the missing module on the build host. If you are using
> bitbake with oe-core, you can also use buildtools" etc.
> 
> If you can give me a canned error message that I can copy paste,
> that'd be best :D

How about:

"""
Unable to use hash equivalence server at '%s' due to missing or
incorrect python module: %s. 

Please install the needed module on the build host, or use an
environment containing it (e.g a pip venv or OE's buildtools tarball)
or disable the BB_HASHSERVE_UPSTREAM setting.
"""


Cheers,

Richard