| Message ID | 20251208161219.41335-1-corentin.guillevic@smile.fr |
|---|---|
| State | New |
| Headers | show |
| Series | fetch2: Skip AUTOREV management if no SCM URI | expand |
On Mon, 2025-12-08 at 17:12 +0100, Corentin Guillevic via lists.openembedded.org wrote: > AUTOREV allows to fetch the latest revision. If no SCM URI is used (e.g. git://, ...), > AUTOREV has no effect. However there is one case in which a recipe with AUTOREV > but no SCM URI will lead to a parsing error: > > ERROR: /path/to/yocto/builds/build-qemux86-64/workspace/recipes/i2c-tools/i2c-tools_3.1.2.bb: AUTOREV/SRCPV set too late for the fetcher to work properly, please set the variables earlier in parsing. Erroring instead of later obtuse build failures. > > This error is raised when AUTOREV occurs (__BBAUTOREV_SEEN) but is not acted > upon (__BBAUTOREV_ACTED_UPON). A recipe with one SCM URI causes the > srcrev_internal_helper() function to run, and read the value of SRCREV. If > SRCREV = "${AUTOREV}", then AUTOREV will be acted upon. > > If the recipe does not have an SCM URI, SRCREV is not parsed and AUTOREV remains > unchanged. However, if SRCREV is expanded during the parsing time in Python code > (either pure or evaluated from BitBake code), then AUTOREV will be expanded too > and resulting in the setting of __BBAUTOREV_SEEN. However, as there is no SCM > URI, the code flow that would act upon AUTOREV is never run. > > The fix slightly changes the way AUTOREV is handled: if no SCM URI is provided, > AUTOREV will be considered to have never occurred (__BBAUTOREV_SEEN will not be > defined). It makes more sense to me to throw an error if AUTOREV is seen but no SCM URI is provided, rather than silently ignoring AUTOREV. Perhaps we just need to improve the error that is raised. Is there any valid case for `SRCREV = "${AUTOREV}` when no SCM URI is given? Thanks,
> Is there any valid case for `SRCREV = "${AUTOREV}` when no SCM URI is given? No, however this case occurs indirectly with the recipetool command (from the self-test devtool.DevtoolUpgradeTests). It generates a recipe in which SRCREV is equal to "${AUTOREV}", even though the URI is not a SCM one. The issue arose because I used SRCREV in a new "insane" test, see: https://lists.openembedded.org/g/openembedded-core/topic/116341789 The issue can be reproduced by applying my patches and running the following command: $ recipetool --color=auto create --devtool -o /tmp/devtool4819z5wk ' http://downloads.yoctoproject.org/mirror/sources/i2c-tools-3.1.2.tar.bz2' -x ${BUILDDIR}/workspace/sources/devtoolsrcdcviuvn9 Le lun. 8 déc. 2025 à 17:59, Paul Barker <paul@pbarker.dev> a écrit : > On Mon, 2025-12-08 at 17:12 +0100, Corentin Guillevic via > lists.openembedded.org wrote: > > AUTOREV allows to fetch the latest revision. If no SCM URI is used (e.g. > git://, ...), > > AUTOREV has no effect. However there is one case in which a recipe with > AUTOREV > > but no SCM URI will lead to a parsing error: > > > > ERROR: > /path/to/yocto/builds/build-qemux86-64/workspace/recipes/i2c-tools/ > i2c-tools_3.1.2.bb: AUTOREV/SRCPV set too late for the fetcher to work > properly, please set the variables earlier in parsing. Erroring instead of > later obtuse build failures. > > > > This error is raised when AUTOREV occurs (__BBAUTOREV_SEEN) but is not > acted > > upon (__BBAUTOREV_ACTED_UPON). A recipe with one SCM URI causes the > > srcrev_internal_helper() function to run, and read the value of SRCREV. > If > > SRCREV = "${AUTOREV}", then AUTOREV will be acted upon. > > > > If the recipe does not have an SCM URI, SRCREV is not parsed and AUTOREV > remains > > unchanged. However, if SRCREV is expanded during the parsing time in > Python code > > (either pure or evaluated from BitBake code), then AUTOREV will be > expanded too > > and resulting in the setting of __BBAUTOREV_SEEN. However, as there is > no SCM > > URI, the code flow that would act upon AUTOREV is never run. > > > > The fix slightly changes the way AUTOREV is handled: if no SCM URI is > provided, > > AUTOREV will be considered to have never occurred (__BBAUTOREV_SEEN will > not be > > defined). > > It makes more sense to me to throw an error if AUTOREV is seen but no > SCM URI is provided, rather than silently ignoring AUTOREV. Perhaps we > just need to improve the error that is raised. > > Is there any valid case for `SRCREV = "${AUTOREV}` when no SCM URI is > given? > > Thanks, > > -- > Paul Barker > > >
On Tue, 2025-12-09 at 11:43 +0100, Corentin Guillevic via lists.openembedded.org wrote: > > Is there any valid case for `SRCREV = "${AUTOREV}` when no SCM URI is > given? > > No, however this case occurs indirectly with the recipetool command (from > the self-test devtool.DevtoolUpgradeTests). It generates a recipe in which > SRCREV is equal to "${AUTOREV}", even though the URI is not a SCM one. The > issue arose because I used SRCREV in a new "insane" test, see: > https://lists.openembedded.org/g/openembedded-core/topic/116341789 > > The issue can be reproduced by applying my patches and running the > following command: > > $ recipetool --color=auto create --devtool -o /tmp/devtool4819z5wk ' > http://downloads.yoctoproject.org/mirror/sources/i2c-tools-3.1.2.tar.bz2' > -x ${BUILDDIR}/workspace/sources/devtoolsrcdcviuvn9 Which patches are you referring to here? And, does the issue occur on the current master branch without applying any additional patches? > > Le lun. 8 déc. 2025 à 17:59, Paul Barker <paul@pbarker.dev> a écrit : > > > On Mon, 2025-12-08 at 17:12 +0100, Corentin Guillevic via > > lists.openembedded.org wrote: > > > AUTOREV allows to fetch the latest revision. If no SCM URI is used (e.g. > > git://, ...), > > > AUTOREV has no effect. However there is one case in which a recipe with > > AUTOREV > > > but no SCM URI will lead to a parsing error: > > > > > > ERROR: > > /path/to/yocto/builds/build-qemux86-64/workspace/recipes/i2c-tools/ > > i2c-tools_3.1.2.bb: AUTOREV/SRCPV set too late for the fetcher to work > > properly, please set the variables earlier in parsing. Erroring instead of > > later obtuse build failures. > > > > > > This error is raised when AUTOREV occurs (__BBAUTOREV_SEEN) but is not > > acted > > > upon (__BBAUTOREV_ACTED_UPON). A recipe with one SCM URI causes the > > > srcrev_internal_helper() function to run, and read the value of SRCREV. > > If > > > SRCREV = "${AUTOREV}", then AUTOREV will be acted upon. > > > > > > If the recipe does not have an SCM URI, SRCREV is not parsed and AUTOREV > > remains > > > unchanged. However, if SRCREV is expanded during the parsing time in > > Python code > > > (either pure or evaluated from BitBake code), then AUTOREV will be > > expanded too > > > and resulting in the setting of __BBAUTOREV_SEEN. However, as there is > > no SCM > > > URI, the code flow that would act upon AUTOREV is never run. > > > > > > The fix slightly changes the way AUTOREV is handled: if no SCM URI is > > provided, > > > AUTOREV will be considered to have never occurred (__BBAUTOREV_SEEN will > > not be > > > defined). > > > > It makes more sense to me to throw an error if AUTOREV is seen but no > > SCM URI is provided, rather than silently ignoring AUTOREV. Perhaps we > > just need to improve the error that is raised. > > > > Is there any valid case for `SRCREV = "${AUTOREV}` when no SCM URI is > > given? Thanks,
> Which patches are you referring to here? I am referring to these patches: - https://lists.openembedded.org/g/openembedded-core/topic/116341789 - https://lists.openembedded.org/g/openembedded-core/message/226505 They introduce and enable a new "insane" test to check whether if a git URI is provided without a revision (SRCREV). > And, does the issue occur on the current master branch without applying any additional patches? No, the issue doesn't occur on the master branch. This happens with my patches because SRCREV is used in the test, and is expanded during the parsing. If SRCREV is equal to "${AUTOREV}" but no SCM URI is present, AUTOREV will be seen but no SCM function will act upon it. This leads to the error. Le mer. 10 déc. 2025 à 11:20, Paul Barker <paul@pbarker.dev> a écrit : > On Tue, 2025-12-09 at 11:43 +0100, Corentin Guillevic via > lists.openembedded.org wrote: > > > Is there any valid case for `SRCREV = "${AUTOREV}` when no SCM URI is > > given? > > > > No, however this case occurs indirectly with the recipetool command (from > > the self-test devtool.DevtoolUpgradeTests). It generates a recipe in > which > > SRCREV is equal to "${AUTOREV}", even though the URI is not a SCM one. > The > > issue arose because I used SRCREV in a new "insane" test, see: > > https://lists.openembedded.org/g/openembedded-core/topic/116341789 > > > > The issue can be reproduced by applying my patches and running the > > following command: > > > > $ recipetool --color=auto create --devtool -o /tmp/devtool4819z5wk ' > > http://downloads.yoctoproject.org/mirror/sources/i2c-tools-3.1.2.tar.bz2 > ' > > -x ${BUILDDIR}/workspace/sources/devtoolsrcdcviuvn9 > > Which patches are you referring to here? And, does the issue occur on the > current master branch without applying any additional patches? > > > > > Le lun. 8 déc. 2025 à 17:59, Paul Barker <paul@pbarker.dev> a écrit : > > > > > On Mon, 2025-12-08 at 17:12 +0100, Corentin Guillevic via > > > lists.openembedded.org wrote: > > > > AUTOREV allows to fetch the latest revision. If no SCM URI is used > (e.g. > > > git://, ...), > > > > AUTOREV has no effect. However there is one case in which a recipe > with > > > AUTOREV > > > > but no SCM URI will lead to a parsing error: > > > > > > > > ERROR: > > > /path/to/yocto/builds/build-qemux86-64/workspace/recipes/i2c-tools/ > > > i2c-tools_3.1.2.bb: AUTOREV/SRCPV set too late for the fetcher to work > > > properly, please set the variables earlier in parsing. Erroring > instead of > > > later obtuse build failures. > > > > > > > > This error is raised when AUTOREV occurs (__BBAUTOREV_SEEN) but is > not > > > acted > > > > upon (__BBAUTOREV_ACTED_UPON). A recipe with one SCM URI causes the > > > > srcrev_internal_helper() function to run, and read the value of > SRCREV. > > > If > > > > SRCREV = "${AUTOREV}", then AUTOREV will be acted upon. > > > > > > > > If the recipe does not have an SCM URI, SRCREV is not parsed and > AUTOREV > > > remains > > > > unchanged. However, if SRCREV is expanded during the parsing time in > > > Python code > > > > (either pure or evaluated from BitBake code), then AUTOREV will be > > > expanded too > > > > and resulting in the setting of __BBAUTOREV_SEEN. However, as there > is > > > no SCM > > > > URI, the code flow that would act upon AUTOREV is never run. > > > > > > > > The fix slightly changes the way AUTOREV is handled: if no SCM URI is > > > provided, > > > > AUTOREV will be considered to have never occurred (__BBAUTOREV_SEEN > will > > > not be > > > > defined). > > > > > > It makes more sense to me to throw an error if AUTOREV is seen but no > > > SCM URI is provided, rather than silently ignoring AUTOREV. Perhaps we > > > just need to improve the error that is raised. > > > > > > Is there any valid case for `SRCREV = "${AUTOREV}` when no SCM URI is > > > given? > > Thanks, > > -- > Paul Barker > > >
diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index bdd497394..a1e2def95 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -769,8 +769,16 @@ def mark_recipe_nocache(d): d.setVar('BB_DONT_CACHE', '1') def get_autorev(d): - mark_recipe_nocache(d) - d.setVar("__BBAUTOREV_SEEN", True) + fetcher = Fetch(d.getVar('SRC_URI').split(), d) + urldata = fetcher.ud + for u in urldata: + if not urldata[u].method.supports_srcrev(): + continue + + mark_recipe_nocache(d) + d.setVar("__BBAUTOREV_SEEN", True) + break + return "AUTOINC" def _get_srcrev(d, method_name='sortable_revision'):
AUTOREV allows to fetch the latest revision. If no SCM URI is used (e.g. git://, ...), AUTOREV has no effect. However there is one case in which a recipe with AUTOREV but no SCM URI will lead to a parsing error: ERROR: /path/to/yocto/builds/build-qemux86-64/workspace/recipes/i2c-tools/i2c-tools_3.1.2.bb: AUTOREV/SRCPV set too late for the fetcher to work properly, please set the variables earlier in parsing. Erroring instead of later obtuse build failures. This error is raised when AUTOREV occurs (__BBAUTOREV_SEEN) but is not acted upon (__BBAUTOREV_ACTED_UPON). A recipe with one SCM URI causes the srcrev_internal_helper() function to run, and read the value of SRCREV. If SRCREV = "${AUTOREV}", then AUTOREV will be acted upon. If the recipe does not have an SCM URI, SRCREV is not parsed and AUTOREV remains unchanged. However, if SRCREV is expanded during the parsing time in Python code (either pure or evaluated from BitBake code), then AUTOREV will be expanded too and resulting in the setting of __BBAUTOREV_SEEN. However, as there is no SCM URI, the code flow that would act upon AUTOREV is never run. The fix slightly changes the way AUTOREV is handled: if no SCM URI is provided, AUTOREV will be considered to have never occurred (__BBAUTOREV_SEEN will not be defined). Signed-off-by: Corentin Guillevic <corentin.guillevic@smile.fr> --- lib/bb/fetch2/__init__.py | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)