| Message ID | 20260914051148.1445323-1-zhangqiang.zq307@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | fetch: Add log_errors parameter to Fetch.download() | expand |
On Mon, 2026-09-14 at 13:11 +0800, zhangqiang.zq307 via lists.openembedded.org wrote: > From: 张强 <zhangqiang.zq307@gmail.com> > > Fetching an sstate object from a mirror can fail transiently and is > recoverable when the caller falls back to running the real task. > The unconditional logger.error() in download() makes bitbake exit > non-zero even though every task ends up succeeding, as bitbake counts > any ERROR message towards its exit code. > > Add a log_errors parameter so such callers can request the first > fetch failure to be logged at WARNING instead of ERROR. The default > (True) keeps the current behaviour for all existing callers. > > Signed-off-by: 张强 <zhangqiang.zq307@gmail.com> > --- > lib/bb/fetch/__init__.py | 10 ++++++++-- > 1 file changed, 8 insertions(+), 2 deletions(-) There have been a few attempts at changes like this over the years and we've not taken them. If the mirror says something is available, then for some reason it isn't that is an error and should be reported as such, it really needs fixing. Cheers, Richard
I agree an index/object mismatch on the mirror is a real error that needs fixing, and I'm not trying to silence that class of failure. The case I'm hitting is different: the fetch fails with a transient network error (connection reset / timeout between the builder and the mirror), not "the mirror advertised something it doesn't have". The task then falls back to running the real task and succeeds, so nothing actually failed - yet bitbake still exits non-zero purely because of that ERROR line, since ERROR messages count towards the exit code. In CI this turns a fully recovered build into a red one, with no way to tell it apart from a genuine failure. A recovered sstate miss is already a normal, designed-for path; the goal is just to make the exit code reflect the actual outcome. If log_errors is not the right shape, would either of these be more acceptable? - keep ERROR only for "mirror is lying" (404 / checksum mismatch) and log network-level errors (timeout, connection reset) at WARNING, or - have download() raise instead of logging, and let the caller pick the level. Both keep real mirror problems loud while not failing a build over a network blip. Happy to respin the patch in whichever direction you prefer. Qiang, Richard Richard Purdie <richard.purdie@linuxfoundation.org> 于2026年9月14日周一 17:24写道： > On Mon, 2026-09-14 at 13:11 +0800, zhangqiang.zq307 via > lists.openembedded.org wrote: > > From: 张强 <zhangqiang.zq307@gmail.com> > > > > Fetching an sstate object from a mirror can fail transiently and is > > recoverable when the caller falls back to running the real task. > > The unconditional logger.error() in download() makes bitbake exit > > non-zero even though every task ends up succeeding, as bitbake counts > > any ERROR message towards its exit code. > > > > Add a log_errors parameter so such callers can request the first > > fetch failure to be logged at WARNING instead of ERROR. The default > > (True) keeps the current behaviour for all existing callers. > > > > Signed-off-by: 张强 <zhangqiang.zq307@gmail.com> > > --- > > lib/bb/fetch/__init__.py | 10 ++++++++-- > > 1 file changed, 8 insertions(+), 2 deletions(-) > > There have been a few attempts at changes like this over the years and > we've not taken them. If the mirror says something is available, then > for some reason it isn't that is an error and should be reported as > such, it really needs fixing. > > Cheers, > > Richard >
On Mon, 14 Sept 2026 at 12:04, 张强 via lists.openembedded.org <zhangqiang.zq307=gmail.com@lists.openembedded.org> wrote: > Both keep real mirror problems loud while not failing a build over a > network blip. Happy to respin the patch in whichever direction you > prefer. 'Network blips' are just as real problems as missing cache objects. They're not 'problems that can be recovered from', they're infrastructure errors that need fixing and should be reported as such, regardless of whether the build process can find a way to produce the needed artefacts in the end. Alex
Understood, thanks both. I'll look into the mirror/network side and handle the exit-code concern on our infrastructure instead. Alexander Kanavin <alex.kanavin@gmail.com> 于2026年9月14日周一 18:20写道： > On Mon, 14 Sept 2026 at 12:04, 张强 via lists.openembedded.org > <zhangqiang.zq307=gmail.com@lists.openembedded.org> wrote: > > Both keep real mirror problems loud while not failing a build over a > > network blip. Happy to respin the patch in whichever direction you > > prefer. > > 'Network blips' are just as real problems as missing cache objects. > They're not 'problems that can be recovered from', they're > infrastructure errors that need fixing and should be reported as such, > regardless of whether the build process can find a way to produce the > needed artefacts in the end. > > Alex >
diff --git a/lib/bb/fetch/__init__.py b/lib/bb/fetch/__init__.py index 55ab710f3..0c3a4a7d4 100644 --- a/lib/bb/fetch/__init__.py +++ b/lib/bb/fetch/__init__.py @@ -1884,9 +1884,12 @@ class Fetch(object): return local - def download(self, urls=None): + def download(self, urls=None, log_errors=True): """ Fetch all urls + + log_errors=False logs the first fetch failure at WARNING instead of + ERROR, for callers that treat a fetch failure as recoverable. """ if not urls: urls = self.urls @@ -1957,7 +1960,10 @@ class Fetch(object): if not done or not m.done(ud, d): if firsterr: - logger.error(str(firsterr)) + if log_errors: + logger.error(str(firsterr)) + else: + logger.warning(str(firsterr)) raise FetchError("Unable to fetch URL from any source.", u) m.update_donestamp(ud, d)