| Message ID | 20241005001606.51818-1-yumx@amazon.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | sstate: remove corrupted artifacts from local mirror | expand |
On Fri, Oct 4, 2024 at 5:16 PM Yu, Max via lists.yoctoproject.org <yumx=amazon.com@lists.yoctoproject.org> wrote: > > We observe sstate cache corruptions sometimes which cause rebuilds. That is not > a fatal error as the package has to be rebuilt and updated artifact needs to be > pushed to remote sstate cache mirror. Currently, Yocto does not handle > corruptions properly, where the corrupted artifact is not deleted or > renamed. Later, after the package is built the same corrupted artifact is pushed > to remote mirror and the same procedure is circled again and again. > > This change verifies the outcome of the unpacking action and renames the > artifact if a fatal error occurred ("tar" tool returns error 2). In such case we > rename the artifact what causes that a proper one is created and uploaded > overwriting the exisiting one - the corrupted one - in the remote mirror. That > way we break the loop of uploading corrupted file again and again. > Thanks for contributing and trying to make the yocto project better. I think the patch seems ok to me. However, please send it to openembedded-core mailing list openembedded-core@lists.openembedded.org since this is part of oe-core which poky uses. I would also suggest to peruse through - https://docs.yoctoproject.org/dev/contributor-guide/submit-changes.html > Suggested-by: Przemyslaw Sobon <psobon@amazon.com> > Signed-off-by: Max Yu <yumx@amazon.com> > --- > meta/classes-global/sstate.bbclass | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass > index 11bb892a42..5a7ce35341 100644 > --- a/meta/classes-global/sstate.bbclass > +++ b/meta/classes-global/sstate.bbclass > @@ -937,7 +937,12 @@ sstate_unpack_package () { > ZSTD="pzstd -p ${ZSTD_THREADS}" > fi > > - tar -I "$ZSTD" -xvpf ${SSTATE_PKG} > + if ! tar -I "$ZSTD" -xvpf ${SSTATE_PKG}; then > + echo "Fatal error extracting sstate cache artifacts, file might be corrupted or truncated, renaming" > + mv ${SSTATE_PKG} ${SSTATE_PKG}.unpack_error > + exit 2 > + fi > + > # update .siginfo atime on local/NFS mirror if it is a symbolic link > [ ! -h ${SSTATE_PKG}.siginfo ] || [ ! -e ${SSTATE_PKG}.siginfo ] || touch -a ${SSTATE_PKG}.siginfo 2>/dev/null || true > # update each symbolic link instead of any referenced file > -- > 2.40.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#13449): https://lists.yoctoproject.org/g/poky/message/13449 > Mute This Topic: https://lists.yoctoproject.org/mt/108827961/1997914 > Group Owner: poky+owner@lists.yoctoproject.org > Unsubscribe: https://lists.yoctoproject.org/g/poky/unsub [raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Gotcha, will do, thanks! Max On 10/4/24, 5:30 PM, "poky@lists.yoctoproject.org <mailto:poky@lists.yoctoproject.org> on behalf of Khem Raj" <poky@lists.yoctoproject.org <mailto:poky@lists.yoctoproject.org> on behalf of raj.khem@gmail.com <mailto:raj.khem@gmail.com>> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. On Fri, Oct 4, 2024 at 5:16 PM Yu, Max via lists.yoctoproject.org <yumx=amazon.com@lists.yoctoproject.org <mailto:amazon.com@lists.yoctoproject.org>> wrote: > > We observe sstate cache corruptions sometimes which cause rebuilds. That is not > a fatal error as the package has to be rebuilt and updated artifact needs to be > pushed to remote sstate cache mirror. Currently, Yocto does not handle > corruptions properly, where the corrupted artifact is not deleted or > renamed. Later, after the package is built the same corrupted artifact is pushed > to remote mirror and the same procedure is circled again and again. > > This change verifies the outcome of the unpacking action and renames the > artifact if a fatal error occurred ("tar" tool returns error 2). In such case we > rename the artifact what causes that a proper one is created and uploaded > overwriting the exisiting one - the corrupted one - in the remote mirror. That > way we break the loop of uploading corrupted file again and again. > Thanks for contributing and trying to make the yocto project better. I think the patch seems ok to me. However, please send it to openembedded-core mailing list openembedded-core@lists.openembedded.org <mailto:openembedded-core@lists.openembedded.org> since this is part of oe-core which poky uses. I would also suggest to peruse through - https://docs.yoctoproject.org/dev/contributor-guide/submit-changes.html <https://docs.yoctoproject.org/dev/contributor-guide/submit-changes.html> > Suggested-by: Przemyslaw Sobon <psobon@amazon.com <mailto:psobon@amazon.com>> > Signed-off-by: Max Yu <yumx@amazon.com <mailto:yumx@amazon.com>> > --- > meta/classes-global/sstate.bbclass | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass > index 11bb892a42..5a7ce35341 100644 > --- a/meta/classes-global/sstate.bbclass > +++ b/meta/classes-global/sstate.bbclass > @@ -937,7 +937,12 @@ sstate_unpack_package () { > ZSTD="pzstd -p ${ZSTD_THREADS}" > fi > > - tar -I "$ZSTD" -xvpf ${SSTATE_PKG} > + if ! tar -I "$ZSTD" -xvpf ${SSTATE_PKG}; then > + echo "Fatal error extracting sstate cache artifacts, file might be corrupted or truncated, renaming" > + mv ${SSTATE_PKG} ${SSTATE_PKG}.unpack_error > + exit 2 > + fi > + > # update .siginfo atime on local/NFS mirror if it is a symbolic link > [ ! -h ${SSTATE_PKG}.siginfo ] || [ ! -e ${SSTATE_PKG}.siginfo ] || touch -a ${SSTATE_PKG}.siginfo 2>/dev/null || true > # update each symbolic link instead of any referenced file > -- > 2.40.1 > > > >
diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass index 11bb892a42..5a7ce35341 100644 --- a/meta/classes-global/sstate.bbclass +++ b/meta/classes-global/sstate.bbclass @@ -937,7 +937,12 @@ sstate_unpack_package () { ZSTD="pzstd -p ${ZSTD_THREADS}" fi - tar -I "$ZSTD" -xvpf ${SSTATE_PKG} + if ! tar -I "$ZSTD" -xvpf ${SSTATE_PKG}; then + echo "Fatal error extracting sstate cache artifacts, file might be corrupted or truncated, renaming" + mv ${SSTATE_PKG} ${SSTATE_PKG}.unpack_error + exit 2 + fi + # update .siginfo atime on local/NFS mirror if it is a symbolic link [ ! -h ${SSTATE_PKG}.siginfo ] || [ ! -e ${SSTATE_PKG}.siginfo ] || touch -a ${SSTATE_PKG}.siginfo 2>/dev/null || true # update each symbolic link instead of any referenced file
We observe sstate cache corruptions sometimes which cause rebuilds. That is not a fatal error as the package has to be rebuilt and updated artifact needs to be pushed to remote sstate cache mirror. Currently, Yocto does not handle corruptions properly, where the corrupted artifact is not deleted or renamed. Later, after the package is built the same corrupted artifact is pushed to remote mirror and the same procedure is circled again and again. This change verifies the outcome of the unpacking action and renames the artifact if a fatal error occurred ("tar" tool returns error 2). In such case we rename the artifact what causes that a proper one is created and uploaded overwriting the exisiting one - the corrupted one - in the remote mirror. That way we break the loop of uploading corrupted file again and again. Suggested-by: Przemyslaw Sobon <psobon@amazon.com> Signed-off-by: Max Yu <yumx@amazon.com> --- meta/classes-global/sstate.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)