mbox series

[RFC,0/4] Clean AUTOREV usage, add revision check for every SCM URI

Message ID 20260304104342.869457-1-corentin.guillevic@smile.fr
Headers show
Series Clean AUTOREV usage, add revision check for every SCM URI | expand

Message

Corentin Guillevic March 4, 2026, 10:43 a.m. UTC 
This patch series introduces an additional "insane" test to check whether every
URI within the SRC_URI variable has a revision. This is the purpose of the last
two patches.

However, the new test raises a "deep" issue when AUTOREV is set:
"AUTOREV/SRCPV set too late for the fetcher to work properly". This occurs
because the test accesses to the SRCREV variable, at the line:

if d.getVar('SRCREV') == "INVALID":

The following self-tests can raise this issue (if my test is enabled):

$ oe-selftest -Kv -r bblayers.BitbakeLayers \
    -r pokybleeding.PokyBleeding.test_poky_bleeding_autorev \
    -r devtool.DevtoolUpgradeTests

Or a shorter one:

$ recipetool --color=auto create --devtool -o /tmp/devtool4819z5wk \
    'http://downloads.yoctoproject.org/mirror/sources/i2c-tools-3.1.2.tar.bz2' \
    -x ${BUILDDIR}/workspace/sources/devtoolsrcdcviuvn9

The common point is AUTOREV, which is used as default value by devtool and
recipetool. When AUTOREV is parsed, this triggers the call to the get_autorev()
function (see [1]) which sets the internal __BBAUTOREV_SEEN variable.

If the associated URI is a SCM one, there is no problem. Otherwise, the above
error will occur (see [2]) because the fetcher code doesn't take AUTOREV into
account (see [3]).

Why has this error never occurred before? In my opinion, it is related to how
devtool/recipetool parse and run their crafted recipes. Firstly, these tools
unconditionally set AUTOREV as default value for srcrev. Despite this value is
not written into the temporary recipe if SRC_URI doesn't have a SCM URI, the
tools seem to keep it in memory.

Next, the crafted recipe is run. Without my patch series, no error is raised
because I think there is no use of SRCREV in the execution flow. With my insane
test only, which uses SRCREV, the test code is parsed and expanded. As SRCREV
is being used, its value AUTOREV is expanded and leads to function
get_autorev(). As the URI is no a SCM one, no code handles AUTOREV and the
above error occurs.

This is why the first two patches: their purpose is to set srcrev to AUTOREV
only if the submitted URI is a SCM one. These patches resolve, the
"AUTOREV/SRCPV set too late for the fetcher to work properly" error disappears,
as AUTOREV is only set when required.

I sent this patch series as RFC because I need feedback on this issue. What
are your thoughts on the error and my fix? Is my theory correct about
devtool/recipetool keeping some variables in memory before running the crafted
recipe?

Thank you

Corentin

- [1] https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/__init__.py
- [2] Function finalize() from https://git.openembedded.org/bitbake/tree/lib/bb/parse/ast.py
- [3] Function setup_revisions() from https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/__init__.py, only called from SCM fetchers

Corentin Guillevic (4):
  insane.bbclass: check revision for every git URI
  layer.conf: enable test for revision alongside a git URI
  recipetool: create: use default AUTOREV only for SCM URI
  devtool: use default AUTOREV only for SCM URI

 meta/classes-global/insane.bbclass | 27 +++++++++++++++++++++++++++
 meta/conf/layer.conf               |  3 +++
 scripts/lib/devtool/upgrade.py     |  7 ++++++-
 scripts/lib/recipetool/create.py   |  7 ++++++-
 4 files changed, 42 insertions(+), 2 deletions(-)