| Message ID | 20250624104155.559827-9-yi.zhao@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | fix runtime versions | expand |
On Tue, 2025-06-24 at 18:41 +0800, Yi Zhao via lists.openembedded.org wrote: > The runtime version of psmisc is generated by 'git describe' command in > misc/git-version-gen. But when using git shallow tarball, > git-version-gen can not get the correct version due to missing git > tag/commit messages. This actually breaks reproducibility, since the > generated binaries should be consistent regardless of the source code > packaging format. > > Add a .tarball-version file in source directory. Then get-version-gen > can determine the version through this file instead of 'git describe' > command. > > Before the fix: > root@intel-x86-64:~# pslog -V > pslog (PSmisc) UNKNOWN > > After the fix: > root@intel-x86-64:~# pslog -V > pslog (PSmisc) 23.7 > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > --- > meta/recipes-extended/psmisc/psmisc_23.7.bb | 1 + > 1 file changed, 1 insertion(+) Does adding the tag parameter to the url fix this? That should cause the shallow clones to fetch the tag data. Cheers, Richard
On 6/25/25 16:19, Richard Purdie wrote: > On Tue, 2025-06-24 at 18:41 +0800, Yi Zhao via lists.openembedded.org wrote: >> The runtime version of psmisc is generated by 'git describe' command in >> misc/git-version-gen. But when using git shallow tarball, >> git-version-gen can not get the correct version due to missing git >> tag/commit messages. This actually breaks reproducibility, since the >> generated binaries should be consistent regardless of the source code >> packaging format. >> >> Add a .tarball-version file in source directory. Then get-version-gen >> can determine the version through this file instead of 'git describe' >> command. >> >> Before the fix: >> root@intel-x86-64:~# pslog -V >> pslog (PSmisc) UNKNOWN >> >> After the fix: >> root@intel-x86-64:~# pslog -V >> pslog (PSmisc) 23.7 >> >> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> >> --- >> meta/recipes-extended/psmisc/psmisc_23.7.bb | 1 + >> 1 file changed, 1 insertion(+) > Does adding the tag parameter to the url fix this? I have tested it. But it doesn't work. //Yi > > That should cause the shallow clones to fetch the tag data. > > Cheers, > > Richard
On Wed, 2025-06-25 at 18:22 +0800, Yi Zhao wrote: > > On 6/25/25 16:19, Richard Purdie wrote: > > On Tue, 2025-06-24 at 18:41 +0800, Yi Zhao via > > lists.openembedded.org wrote: > > > The runtime version of psmisc is generated by 'git describe' > > > command in > > > misc/git-version-gen. But when using git shallow tarball, > > > git-version-gen can not get the correct version due to missing > > > git > > > tag/commit messages. This actually breaks reproducibility, since > > > the > > > generated binaries should be consistent regardless of the source > > > code > > > packaging format. > > > > > > Add a .tarball-version file in source directory. Then get- > > > version-gen > > > can determine the version through this file instead of 'git > > > describe' > > > command. > > > > > > Before the fix: > > > root@intel-x86-64:~# pslog -V > > > pslog (PSmisc) UNKNOWN > > > > > > After the fix: > > > root@intel-x86-64:~# pslog -V > > > pslog (PSmisc) 23.7 > > > > > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > > > --- > > > meta/recipes-extended/psmisc/psmisc_23.7.bb | 1 + > > > 1 file changed, 1 insertion(+) > > Does adding the tag parameter to the url fix this? > > I have tested it. But it doesn't work. I was curious why not and I had a look myself. The trouble is that the way the shallow cloning works, it squashes out references so "v23.7" becomes just that, not "refs/tags/v23.7". I'm not 100% convinced the fetcher is doing the right thing here. If we could convince the fetcher to preserve the refs/tags piece, then this probably would work... At the very least I wanted to document why it didn't work. After writing this, I did dig a bit further. If I change the fetcher like this: diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py index 0af8070080b..e7b030998c0 100644 --- a/bitbake/lib/bb/fetch2/git.py +++ b/bitbake/lib/bb/fetch2/git.py @@ -634,7 +634,7 @@ class Git(FetchMethod): all_refs.append(line.split()[-1]) extra_refs = [] if 'tag' in ud.parm: - extra_refs.append(ud.parm['tag']) + extra_refs.append("refs/tags/" + ud.parm['tag']) for r in ud.shallow_extra_refs: if not ud.bareclone: r = r.replace('refs/heads/', 'refs/remotes/origin/') it does work. Perhaps we should do that. Cheers, Richard
On Wed, 2025-06-25 at 20:04 +0100, Richard Purdie via lists.openembedded.org wrote: > On Wed, 2025-06-25 at 18:22 +0800, Yi Zhao wrote: > > > > On 6/25/25 16:19, Richard Purdie wrote: > > > On Tue, 2025-06-24 at 18:41 +0800, Yi Zhao via > > > lists.openembedded.org wrote: > > > > The runtime version of psmisc is generated by 'git describe' > > > > command in > > > > misc/git-version-gen. But when using git shallow tarball, > > > > git-version-gen can not get the correct version due to missing > > > > git > > > > tag/commit messages. This actually breaks reproducibility, > > > > since > > > > the > > > > generated binaries should be consistent regardless of the > > > > source > > > > code > > > > packaging format. > > > > > > > > Add a .tarball-version file in source directory. Then get- > > > > version-gen > > > > can determine the version through this file instead of 'git > > > > describe' > > > > command. > > > > > > > > Before the fix: > > > > root@intel-x86-64:~# pslog -V > > > > pslog (PSmisc) UNKNOWN > > > > > > > > After the fix: > > > > root@intel-x86-64:~# pslog -V > > > > pslog (PSmisc) 23.7 > > > > > > > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > > > > --- > > > > meta/recipes-extended/psmisc/psmisc_23.7.bb | 1 + > > > > 1 file changed, 1 insertion(+) > > > Does adding the tag parameter to the url fix this? > > > > I have tested it. But it doesn't work. > > I was curious why not and I had a look myself. The trouble is that > the > way the shallow cloning works, it squashes out references so "v23.7" > becomes just that, not "refs/tags/v23.7". > > I'm not 100% convinced the fetcher is doing the right thing here. If > we > could convince the fetcher to preserve the refs/tags piece, then this > probably would work... > > At the very least I wanted to document why it didn't work. > > After writing this, I did dig a bit further. If I change the fetcher > like this: > > diff --git a/bitbake/lib/bb/fetch2/git.py > b/bitbake/lib/bb/fetch2/git.py > index 0af8070080b..e7b030998c0 100644 > --- a/bitbake/lib/bb/fetch2/git.py > +++ b/bitbake/lib/bb/fetch2/git.py > @@ -634,7 +634,7 @@ class Git(FetchMethod): > all_refs.append(line.split()[-1]) > extra_refs = [] > if 'tag' in ud.parm: > - extra_refs.append(ud.parm['tag']) > + extra_refs.append("refs/tags/" + ud.parm['tag']) > for r in ud.shallow_extra_refs: > if not ud.bareclone: > r = r.replace('refs/heads/', 'refs/remotes/origin/') > > > it does work. Perhaps we should do that. I've sent a couple of patches doing this, and adding the tag to the shallow tarball name so we can add tags to recipes without complaints and migrate to the new layout without issue. Cheers, Richard
diff --git a/meta/recipes-extended/psmisc/psmisc_23.7.bb b/meta/recipes-extended/psmisc/psmisc_23.7.bb index fff1f218f4..a53c27017d 100644 --- a/meta/recipes-extended/psmisc/psmisc_23.7.bb +++ b/meta/recipes-extended/psmisc/psmisc_23.7.bb @@ -23,6 +23,7 @@ inherit autotools gettext # doesn't believe po/ is a gettext directory and won't generate po/Makefile. do_configure:prepend() { ( cd ${S} && po/update-potfiles ) + [ ! -e ${S}/.tarball-version ] && echo ${PV} > ${S}/.tarball-version } PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)}"
The runtime version of psmisc is generated by 'git describe' command in misc/git-version-gen. But when using git shallow tarball, git-version-gen can not get the correct version due to missing git tag/commit messages. This actually breaks reproducibility, since the generated binaries should be consistent regardless of the source code packaging format. Add a .tarball-version file in source directory. Then get-version-gen can determine the version through this file instead of 'git describe' command. Before the fix: root@intel-x86-64:~# pslog -V pslog (PSmisc) UNKNOWN After the fix: root@intel-x86-64:~# pslog -V pslog (PSmisc) 23.7 Signed-off-by: Yi Zhao <yi.zhao@windriver.com> --- meta/recipes-extended/psmisc/psmisc_23.7.bb | 1 + 1 file changed, 1 insertion(+)