mbox series

[0/1] Subject: [RFC PATCH] bitbake: fetch2/git: add switch to disable fast shallow path

Message ID 20260430123847.25046-1-marcio.henriques@ctw.bmwgroup.com
Headers show
Series Subject: [RFC PATCH] bitbake: fetch2/git: add switch to disable fast shallow path | expand

Message

Marcio Henriques April 30, 2026, 12:38 p.m. UTC 
Hi all,

This is an RFC for a small behavior change in the Git fetcher shallow flow.

Problem statement:

Our CI setup uses a hybrid strategy combining both a full local git clone and shallow tarballs.

The full bare clone in DL_DIR/git2/ is the primary working copy. It is kept up to date across builds so that each CI run only needs to fetch the delta of new commits rather than re-cloning from scratch. This is used to bump revisions.

Shallow tarballs serve as fallback artifacts for cases where an upstream revision has disappeared (e.g. due to a force push or a deleted branch). They also allow DL_DIR to be cleaned up at any time, since all previously integrated revisions remain available as shallow tarballs and can be used to restore a working state without fetching the full upstream history.

The problem is that enabling BB_GIT_SHALLOW also silently activates a fast shallow path that, when no local clone exists, clones directly from upstream into a temporary directory and produces a shallow tarball — completely bypassing creation of a persistent full clone in DL_DIR/git2/.

Proposed change:
Introduce a new configuration variable, BB_GIT_SHALLOW_SKIP_FAST.
When set, shallow mode remains fully functional (tarballs are still generated and used as fallback) but the fetcher always goes through the regular clone/update path first, ensuring DL_DIR/git2/ is created and kept current. Subsequent builds fetch only the delta.

Why RFC:
I want feedback on:
1. Variable naming and scope.
2. Whether this behavior should remain opt-in.
3. Any concerns about interaction with existing shallow tarball workflows.

Compatibility:
Default behavior is unchanged unless BB_GIT_SHALLOW_SKIP_FAST is explicitly set to 1.

Expected benefits:
1. Better CI network efficiency for repeated builds.
2. Keep shallow tarballs as fallback artifacts rather than primary source.
3. More consistent use of local mirror/clone state in DL_DIR.

Configuration example:
BB_GIT_SHALLOW = "1"
BB_GIT_SHALLOW_SKIP_FAST = "1"

Thanks for feedback.

Best regards,
Marcio Henriques

Marcio Henriques (1):
  bitbake: add BB_GIT_SHALLOW_SKIP_FAST to disable fast shallow mode

 .../bitbake-user-manual-ref-variables.rst     | 19 +++++++++++++++++++
 lib/bb/fetch2/git.py                          |  4 +++-
 2 files changed, 22 insertions(+), 1 deletion(-)