| Message ID | 20250516093127.506158-1-yi.zhao@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/2] procps: fix runtime version when using git shallow tarball | expand |
Le ven. 16 mai 2025 à 11:31, Yi Zhao via lists.openembedded.org <yi.zhao= eng.windriver.com@lists.openembedded.org> a écrit : > The runtime version of procps is generated by 'git describe' command in > local/git-version-gen. But when using git shallow tarball, > git-version-gen can not get the correct version due to missing git > tag/commit messages. This actually breaks reproducibility, since the > generated binaries should be consistent regardless of the source code > packaging format. > > Add a .tarball-version file in source directory. Then get-version-gen > can determine the version through this file instead of 'git describe' > command. > > Before the fix: > root@intel-x86-64:~# ps -V > ps from procps-ng UNKNOWN > root@intel-x86-64:~# sysctl -V > sysctl from procps-ng UNKNOWN > > After the fix: > root@intel-x86-64:~# ps -V > ps from procps-ng 4.0.5 > root@intel-x86-64:~# sysctl -V > sysctl from procps-ng 4.0.5 > Hello, Did you try to add the tag parameter to SRC_URI? Since https://git.openembedded.org/bitbake/commit/?id=03945475886b8d31c7eccc80fe594ab5306bcf5d, it looks like it should work. > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > --- > meta/recipes-extended/procps/procps_4.0.5.bb | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/meta/recipes-extended/procps/procps_4.0.5.bb > b/meta/recipes-extended/procps/procps_4.0.5.bb > index 3be40c38a4..e3f5bf6b65 100644 > --- a/meta/recipes-extended/procps/procps_4.0.5.bb > +++ b/meta/recipes-extended/procps/procps_4.0.5.bb > @@ -31,6 +31,10 @@ EXTRA_OECONF = "--enable-skill --disable-modern-top" > PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'systemd', d)}" > PACKAGECONFIG[systemd] = "--with-systemd,--without-systemd,systemd" > > +do_configure:prepend () { > + [ ! -e ${S}/.tarball-version ] && echo ${PV} > > ${S}/.tarball-version > +} > + > do_install:append () { > install -d ${D}${base_bindir} > [ "${bindir}" != "${base_bindir}" ] && for i in > ${base_bindir_progs}; do mv ${D}${bindir}/$i ${D}${base_bindir}/$i; done > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#216731): > https://lists.openembedded.org/g/openembedded-core/message/216731 > Mute This Topic: https://lists.openembedded.org/mt/113142319/4316185 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > yoann.congal@smile.fr] > -=-=-=-=-=-=-=-=-=-=-=- > >
> -----Original Message----- > From: openembedded-core@lists.openembedded.org <openembedded- > core@lists.openembedded.org> On Behalf Of Yoann Congal via > lists.openembedded.org > Sent: Friday, May 16, 2025 11:36 > To: yi.zhao@eng.windriver.com > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH 1/2] procps: fix runtime version when using git > shallow tarball > > > > Le ven. 16 mai 2025 à 11:31, Yi Zhao via lists.openembedded.org > <http://lists.openembedded.org> > <yi.zhao=eng.windriver.com@lists.openembedded.org > <mailto:eng.windriver.com@lists.openembedded.org> > a écrit : > > > The runtime version of procps is generated by 'git describe' command in > local/git-version-gen. But when using git shallow tarball, > git-version-gen can not get the correct version due to missing git > tag/commit messages. This actually breaks reproducibility, since the > generated binaries should be consistent regardless of the source code > packaging format. > > Add a .tarball-version file in source directory. Then get-version-gen > can determine the version through this file instead of 'git describe' > command. > > Before the fix: > root@intel-x86-64:~# ps -V > ps from procps-ng UNKNOWN > root@intel-x86-64:~# sysctl -V > sysctl from procps-ng UNKNOWN > > After the fix: > root@intel-x86-64:~# ps -V > ps from procps-ng 4.0.5 > root@intel-x86-64:~# sysctl -V > sysctl from procps-ng 4.0.5 > > > > Hello, > Did you try to add the tag parameter to SRC_URI? > Since > https://git.openembedded.org/bitbake/commit/?id=03945475886b8d31c7eccc80fe > 594ab5306bcf5d, it looks like it should work. There is one problem with the tag in SRC_URI - it should be done only on upgrade. If you just add tag without bumping the version, you will still get the shallow tarball without the tag from already populated local download folder or download mirror. Peter > > > > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com > <mailto:yi.zhao@windriver.com> > > --- > meta/recipes-extended/procps/procps_4.0.5.bb <http://procps_4.0.5.bb> > | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/meta/recipes-extended/procps/procps_4.0.5.bb > <http://procps_4.0.5.bb> b/meta/recipes-extended/procps/procps_4.0.5.bb > <http://procps_4.0.5.bb> > index 3be40c38a4..e3f5bf6b65 100644 > --- a/meta/recipes-extended/procps/procps_4.0.5.bb > <http://procps_4.0.5.bb> > +++ b/meta/recipes-extended/procps/procps_4.0.5.bb > <http://procps_4.0.5.bb> > @@ -31,6 +31,10 @@ EXTRA_OECONF = "--enable-skill --disable-modern- > top" > PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', > 'systemd', d)}" > PACKAGECONFIG[systemd] = "--with-systemd,--without- > systemd,systemd" > > +do_configure:prepend () { > + [ ! -e ${S}/.tarball-version ] && echo ${PV} > ${S}/.tarball-version > +} > + > do_install:append () { > install -d ${D}${base_bindir} > [ "${bindir}" != "${base_bindir}" ] && for i in ${base_bindir_progs}; do > mv ${D}${bindir}/$i ${D}${base_bindir}/$i; done > -- > 2.34.1 > > > > > > > > > -- > > Yoann Congal > Smile ECS
Le ven. 16 mai 2025 à 11:44, Marko, Peter <Peter.Marko@siemens.com> a écrit : > > > > -----Original Message----- > > From: openembedded-core@lists.openembedded.org <openembedded- > > core@lists.openembedded.org> On Behalf Of Yoann Congal via > > lists.openembedded.org > > Sent: Friday, May 16, 2025 11:36 > > To: yi.zhao@eng.windriver.com > > Cc: openembedded-core@lists.openembedded.org > > Subject: Re: [OE-core] [PATCH 1/2] procps: fix runtime version when > using git > > shallow tarball > > > > > > > > Le ven. 16 mai 2025 à 11:31, Yi Zhao via lists.openembedded.org > > <http://lists.openembedded.org> > > <yi.zhao=eng.windriver.com@lists.openembedded.org > > <mailto:eng.windriver.com@lists.openembedded.org> > a écrit : > > > > > > The runtime version of procps is generated by 'git describe' > command in > > local/git-version-gen. But when using git shallow tarball, > > git-version-gen can not get the correct version due to missing git > > tag/commit messages. This actually breaks reproducibility, since > the > > generated binaries should be consistent regardless of the source > code > > packaging format. > > > > Add a .tarball-version file in source directory. Then > get-version-gen > > can determine the version through this file instead of 'git > describe' > > command. > > > > Before the fix: > > root@intel-x86-64:~# ps -V > > ps from procps-ng UNKNOWN > > root@intel-x86-64:~# sysctl -V > > sysctl from procps-ng UNKNOWN > > > > After the fix: > > root@intel-x86-64:~# ps -V > > ps from procps-ng 4.0.5 > > root@intel-x86-64:~# sysctl -V > > sysctl from procps-ng 4.0.5 > > > > > > > > Hello, > > Did you try to add the tag parameter to SRC_URI? > > Since > > > https://git.openembedded.org/bitbake/commit/?id=03945475886b8d31c7eccc80fe > > 594ab5306bcf5d, it looks like it should work. > > There is one problem with the tag in SRC_URI - it should be done only on > upgrade. > If you just add tag without bumping the version, you will still get the > shallow tarball without the tag from already populated local download > folder or download mirror. > Ok, I did not know. Sounds like a bug in the git fetcher. Maybe the clonedir_need_update function needs to be upgraded to handle this case? https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/git.py#n317 > Peter > > > > > > > > > > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com > > <mailto:yi.zhao@windriver.com> > > > --- > > meta/recipes-extended/procps/procps_4.0.5.bb < > http://procps_4.0.5.bb> > > | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/meta/recipes-extended/procps/procps_4.0.5.bb > > <http://procps_4.0.5.bb> b/meta/recipes-extended/procps/procps_4.0.5.bb > > <http://procps_4.0.5.bb> > > index 3be40c38a4..e3f5bf6b65 100644 > > --- a/meta/recipes-extended/procps/procps_4.0.5.bb > > <http://procps_4.0.5.bb> > > +++ b/meta/recipes-extended/procps/procps_4.0.5.bb > > <http://procps_4.0.5.bb> > > @@ -31,6 +31,10 @@ EXTRA_OECONF = "--enable-skill --disable-modern- > > top" > > PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', > > 'systemd', d)}" > > PACKAGECONFIG[systemd] = "--with-systemd,--without- > > systemd,systemd" > > > > +do_configure:prepend () { > > + [ ! -e ${S}/.tarball-version ] && echo ${PV} > > ${S}/.tarball-version > > +} > > + > > do_install:append () { > > install -d ${D}${base_bindir} > > [ "${bindir}" != "${base_bindir}" ] && for i in > ${base_bindir_progs}; do > > mv ${D}${bindir}/$i ${D}${base_bindir}/$i; done > > -- > > 2.34.1 > > > > > > > > > > > > > > > > > > -- > > > > Yoann Congal > > Smile ECS >
> -----Original Message----- > From: Yoann Congal <yoann.congal@smile.fr> > Sent: Friday, May 16, 2025 12:01 > To: Marko, Peter (FT D EU SK BFS1) <Peter.Marko@siemens.com> > Cc: yi.zhao@eng.windriver.com; openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH 1/2] procps: fix runtime version when using git > shallow tarball > > > > Le ven. 16 mai 2025 à 11:44, Marko, Peter <Peter.Marko@siemens.com > <mailto:Peter.Marko@siemens.com> > a écrit : > > > > > > -----Original Message----- > > From: openembedded-core@lists.openembedded.org > <mailto:openembedded-core@lists.openembedded.org> <openembedded- > > core@lists.openembedded.org <mailto:core@lists.openembedded.org> > > On Behalf Of Yoann Congal via > > lists.openembedded.org <http://lists.openembedded.org> > > Sent: Friday, May 16, 2025 11:36 > > To: yi.zhao@eng.windriver.com <mailto:yi.zhao@eng.windriver.com> > > Cc: openembedded-core@lists.openembedded.org > <mailto:openembedded-core@lists.openembedded.org> > > Subject: Re: [OE-core] [PATCH 1/2] procps: fix runtime version when > using git > > shallow tarball > > > > > > > > Le ven. 16 mai 2025 à 11:31, Yi Zhao via lists.openembedded.org > <http://lists.openembedded.org> > > <http://lists.openembedded.org> > > <yi.zhao=eng.windriver.com@lists.openembedded.org > <mailto:eng.windriver.com@lists.openembedded.org> > > <mailto:eng.windriver.com@lists.openembedded.org > <mailto:eng.windriver.com@lists.openembedded.org> > > a écrit : > > > > > > The runtime version of procps is generated by 'git describe' > command in > > local/git-version-gen. But when using git shallow tarball, > > git-version-gen can not get the correct version due to missing git > > tag/commit messages. This actually breaks reproducibility, since the > > generated binaries should be consistent regardless of the source > code > > packaging format. > > > > Add a .tarball-version file in source directory. Then get-version-gen > > can determine the version through this file instead of 'git describe' > > command. > > > > Before the fix: > > root@intel-x86-64:~# ps -V > > ps from procps-ng UNKNOWN > > root@intel-x86-64:~# sysctl -V > > sysctl from procps-ng UNKNOWN > > > > After the fix: > > root@intel-x86-64:~# ps -V > > ps from procps-ng 4.0.5 > > root@intel-x86-64:~# sysctl -V > > sysctl from procps-ng 4.0.5 > > > > > > > > Hello, > > Did you try to add the tag parameter to SRC_URI? > > Since > > > https://git.openembedded.org/bitbake/commit/?id=03945475886b8d31c7eccc80fe > > 594ab5306bcf5d, it looks like it should work. > > There is one problem with the tag in SRC_URI - it should be done only on > upgrade. > If you just add tag without bumping the version, you will still get the > shallow tarball without the tag from already populated local download folder or > download mirror. > > > > Ok, I did not know. > Sounds like a bug in the git fetcher. > Maybe the clonedir_need_update function needs to be upgraded to handle this > case? > https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/git.py#n317 I'm not sure if it's a good idea to update shallow tarball after downloading it from mirror. One of the use-cases is to verify that upstream is not contacted when downloading from mirror (e.g. for air-gaped environments). If you want this working, better way would probably be to add the tag to filename, so new shallow tarball is generated. Peter > > > Peter > > > > > > > > > > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com > <mailto:yi.zhao@windriver.com> > > <mailto:yi.zhao@windriver.com <mailto:yi.zhao@windriver.com> > > > > --- > > meta/recipes-extended/procps/procps_4.0.5.bb > <http://procps_4.0.5.bb> <http://procps_4.0.5.bb> > > | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/meta/recipes-extended/procps/procps_4.0.5.bb > <http://procps_4.0.5.bb> > > <http://procps_4.0.5.bb> b/meta/recipes- > extended/procps/procps_4.0.5.bb <http://procps_4.0.5.bb> > > <http://procps_4.0.5.bb> > > index 3be40c38a4..e3f5bf6b65 100644 > > --- a/meta/recipes-extended/procps/procps_4.0.5.bb > <http://procps_4.0.5.bb> > > <http://procps_4.0.5.bb> > > +++ b/meta/recipes-extended/procps/procps_4.0.5.bb > <http://procps_4.0.5.bb> > > <http://procps_4.0.5.bb> > > @@ -31,6 +31,10 @@ EXTRA_OECONF = "--enable-skill --disable- > modern- > > top" > > PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', > > 'systemd', d)}" > > PACKAGECONFIG[systemd] = "--with-systemd,--without- > > systemd,systemd" > > > > +do_configure:prepend () { > > + [ ! -e ${S}/.tarball-version ] && echo ${PV} > ${S}/.tarball- > version > > +} > > + > > do_install:append () { > > install -d ${D}${base_bindir} > > [ "${bindir}" != "${base_bindir}" ] && for i in > ${base_bindir_progs}; do > > mv ${D}${bindir}/$i ${D}${base_bindir}/$i; done > > -- > > 2.34.1 > > > > > > > > > > > > > > > > > > -- > > > > Yoann Congal > > Smile ECS > > > > > -- > > Yoann Congal > Smile ECS
diff --git a/meta/recipes-extended/procps/procps_4.0.5.bb b/meta/recipes-extended/procps/procps_4.0.5.bb index 3be40c38a4..e3f5bf6b65 100644 --- a/meta/recipes-extended/procps/procps_4.0.5.bb +++ b/meta/recipes-extended/procps/procps_4.0.5.bb @@ -31,6 +31,10 @@ EXTRA_OECONF = "--enable-skill --disable-modern-top" PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'systemd', d)}" PACKAGECONFIG[systemd] = "--with-systemd,--without-systemd,systemd" +do_configure:prepend () { + [ ! -e ${S}/.tarball-version ] && echo ${PV} > ${S}/.tarball-version +} + do_install:append () { install -d ${D}${base_bindir} [ "${bindir}" != "${base_bindir}" ] && for i in ${base_bindir_progs}; do mv ${D}${bindir}/$i ${D}${base_bindir}/$i; done
The runtime version of procps is generated by 'git describe' command in local/git-version-gen. But when using git shallow tarball, git-version-gen can not get the correct version due to missing git tag/commit messages. This actually breaks reproducibility, since the generated binaries should be consistent regardless of the source code packaging format. Add a .tarball-version file in source directory. Then get-version-gen can determine the version through this file instead of 'git describe' command. Before the fix: root@intel-x86-64:~# ps -V ps from procps-ng UNKNOWN root@intel-x86-64:~# sysctl -V sysctl from procps-ng UNKNOWN After the fix: root@intel-x86-64:~# ps -V ps from procps-ng 4.0.5 root@intel-x86-64:~# sysctl -V sysctl from procps-ng 4.0.5 Signed-off-by: Yi Zhao <yi.zhao@windriver.com> --- meta/recipes-extended/procps/procps_4.0.5.bb | 4 ++++ 1 file changed, 4 insertions(+)