| Message ID | AM0PR02MB576490C2267B6F4B73E297DAA612A@AM0PR02MB5764.eurprd02.prod.outlook.com |
|---|---|
| State | New |
| Headers | show |
| Series | fetch2/repo: add support for fetching submodules | expand |
On Mon, 2025-09-22 at 09:47 +0200, Johannes Kauffmann via lists.openembedded.org wrote: > Add the fetch_submodules parameter, which allows fetching submodules of > projects by passing the --fetch-submodules option to repo sync. > > Signed-off-by: Johannes Kauffmann <johanneskauffmann@hotmail.com> > --- > doc/bitbake-user-manual/bitbake-user-manual-fetching.rst | 4 +++- > lib/bb/fetch2/repo.py | 9 +++++++-- > 2 files changed, 10 insertions(+), 3 deletions(-) This isn't as clear cut as you'd first think. How does this work with mirroring with PREMIRRORS? Does the submodule inforation appear on software manifests? I'm a bit worried that this will allow information into the system which doesn't follow the expected behaviour of fetchers :/ Cheers, Richard
On 9/23/25 10:42, Richard Purdie wrote: > On Mon, 2025-09-22 at 09:47 +0200, Johannes Kauffmann via lists.openembedded.org wrote: >> Add the fetch_submodules parameter, which allows fetching submodules of >> projects by passing the --fetch-submodules option to repo sync. >> >> Signed-off-by: Johannes Kauffmann <johanneskauffmann@hotmail.com> >> --- >> doc/bitbake-user-manual/bitbake-user-manual-fetching.rst | 4 +++- >> lib/bb/fetch2/repo.py | 9 +++++++-- >> 2 files changed, 10 insertions(+), 3 deletions(-) > > This isn't as clear cut as you'd first think. How does this work with > mirroring with PREMIRRORS? Does the submodule inforation appear on > software manifests? > > I'm a bit worried that this will allow information into the system > which doesn't follow the expected behaviour of fetchers :/ > > Cheers, > > Richard Hi, thanks for the review, and sorry for getting back so late. Indeed, this is a bad solution as the submodule info doesn't appear in any manifest or SBOM. I didn't test with PREMIRRORS but I suspect it doesn't work correctly there either. Instead of relying on the submodule declaration in the repo manifest.xml, I'm now using the gitsm:// fetcher in addition to the Repo fetcher. Sorry for the noise. Regards, Johannes
diff --git a/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst index f357765b7..e1781085f 100644 --- a/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst +++ b/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst @@ -658,10 +658,12 @@ This fetcher supports the following parameters: - *"manifest":* Name of the manifest file (default: ``default.xml``). +- *"fetch_submodules":* Whether to fetch Git submodules of a project on the server (default: ``0``). + Here are some example URLs:: SRC_URI = "repo://REPOROOT;protocol=git;branch=some_branch;manifest=my_manifest.xml" - SRC_URI = "repo://REPOROOT;protocol=file;branch=some_branch;manifest=my_manifest.xml" + SRC_URI = "repo://REPOROOT;protocol=file;branch=some_branch;manifest=my_manifest.xml;fetch_submodules=1" .. _az-fetcher: diff --git a/lib/bb/fetch2/repo.py b/lib/bb/fetch2/repo.py index fa4cb8149..de01bcb66 100644 --- a/lib/bb/fetch2/repo.py +++ b/lib/bb/fetch2/repo.py @@ -38,6 +38,7 @@ class Repo(FetchMethod): ud.proto = ud.parm.get('protocol', 'git') ud.branch = ud.parm.get('branch', 'master') ud.manifest = ud.parm.get('manifest', 'default.xml') + ud.fetch_submodules = ud.parm.get('fetch_submodules', '0') == '1' if not ud.manifest.endswith('.xml'): ud.manifest += '.xml' @@ -65,8 +66,12 @@ class Repo(FetchMethod): bb.fetch2.check_network_access(d, "%s init -m %s -b %s -u %s://%s%s%s" % (ud.basecmd, ud.manifest, ud.branch, ud.proto, username, ud.host, ud.path), ud.url) runfetchcmd("%s init -m %s -b %s -u %s://%s%s%s" % (ud.basecmd, ud.manifest, ud.branch, ud.proto, username, ud.host, ud.path), d, workdir=repodir) - bb.fetch2.check_network_access(d, "%s sync %s" % (ud.basecmd, ud.url), ud.url) - runfetchcmd("%s sync" % ud.basecmd, d, workdir=repodir) + sync_cmd = "sync" + if ud.fetch_submodules: + sync_cmd = "%s --fetch-submodules" % sync_cmd + + bb.fetch2.check_network_access(d, "%s %s %s" % (ud.basecmd, sync_cmd, ud.url), ud.url) + runfetchcmd("%s %s" % (ud.basecmd, sync_cmd), d, workdir=repodir) scmdata = ud.parm.get("scmdata", "") if scmdata == "keep":
Add the fetch_submodules parameter, which allows fetching submodules of projects by passing the --fetch-submodules option to repo sync. Signed-off-by: Johannes Kauffmann <johanneskauffmann@hotmail.com> --- doc/bitbake-user-manual/bitbake-user-manual-fetching.rst | 4 +++- lib/bb/fetch2/repo.py | 9 +++++++-- 2 files changed, 10 insertions(+), 3 deletions(-)