| Message ID | 20211201043934.3212357-1-kexin.hao@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | linux-yocto-dev: Make -dev kernel work for a fixed revision | expand |
On Wed, 2021-12-01 at 12:39 +0800, Kevin Hao wrote: > By default the -dev kernel uses the "AUTOREV" to pull in the branch > head as the revision. Some of our BSPs are based on the -dev kernel and > we choose to nail down the kernel to a specific revision when releasing > our product by using some setting like below: > PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev" > SRCREV_machine:pn-linux-yocto-dev = "6fb48ae18a10770702266dd1f1aa500149e361ec" > KBRANCH:pn-linux-yocto-dev = "standard/x86" > LINUX_VERSION = "5.15" > > Since all the standard/* branches will be rebased after each kernel > version bump, we would get bitbake fetch failure due to that specific > commit is not reachable in the new version branch. This kind of issue > can be fixed by setting the "nobranch" parameter in the SRC_URI because > it will cause the fetcher to skip the SHA validation for the branch. > And this also won't cause other side effect because all the branches > will be created in the do_kernel_checkout() and the current branch will > be reset to the reversion we want in do_validate_branches(). > > Signed-off-by: Kevin Hao <kexin.hao@windriver.com> > --- > meta/recipes-kernel/linux/linux-yocto-dev.bb | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb > index 6b6ea9a7e864..7204c3eddc11 100644 > --- a/meta/recipes-kernel/linux/linux-yocto-dev.bb > +++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb > @@ -19,7 +19,8 @@ include recipes-kernel/linux/linux-yocto-dev-revisions.inc > KBRANCH = "standard/base" > KMETA = "kernel-meta" > > -SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine \ > +# Set nobranch to skip the SHA validation for branch if a fixed revesion is used > +SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine;nobranch=1 \ > git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=master;destsuffix=${KMETA}" > > # Set default SRCREVs. Both the machine and meta SRCREVs are statically set I'm afraid this looks to be a bit of a horrible workaround/hack. It happens that if you specify a branch and set nobranch it might do what you want but that certainly isn't by design. Either the revision is in the branch or it isn't. The error is telling you the configuration you set isn't valid and you really need to set a valid configuration, i.e. a revision and a branch or a revision and set nobranch but not both. I'm not sure I understand why the kernel would be rebasing after each kernel release? Is this because it is one of the unversioned branches? I can fix the fetcher to hard error if both are set... Cheers, Richard
On Wed, Dec 1, 2021 at 6:28 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Wed, 2021-12-01 at 12:39 +0800, Kevin Hao wrote: > > By default the -dev kernel uses the "AUTOREV" to pull in the branch > > head as the revision. Some of our BSPs are based on the -dev kernel and > > we choose to nail down the kernel to a specific revision when releasing > > our product by using some setting like below: > > PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev" > > SRCREV_machine:pn-linux-yocto-dev = "6fb48ae18a10770702266dd1f1aa500149e361ec" > > KBRANCH:pn-linux-yocto-dev = "standard/x86" > > LINUX_VERSION = "5.15" > > > > Since all the standard/* branches will be rebased after each kernel > > version bump, we would get bitbake fetch failure due to that specific > > commit is not reachable in the new version branch. This kind of issue > > can be fixed by setting the "nobranch" parameter in the SRC_URI because > > it will cause the fetcher to skip the SHA validation for the branch. > > And this also won't cause other side effect because all the branches > > will be created in the do_kernel_checkout() and the current branch will > > be reset to the reversion we want in do_validate_branches(). > > > > Signed-off-by: Kevin Hao <kexin.hao@windriver.com> > > --- > > meta/recipes-kernel/linux/linux-yocto-dev.bb | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb > > index 6b6ea9a7e864..7204c3eddc11 100644 > > --- a/meta/recipes-kernel/linux/linux-yocto-dev.bb > > +++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb > > @@ -19,7 +19,8 @@ include recipes-kernel/linux/linux-yocto-dev-revisions.inc > > KBRANCH = "standard/base" > > KMETA = "kernel-meta" > > > > -SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine \ > > +# Set nobranch to skip the SHA validation for branch if a fixed revesion is used > > +SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine;nobranch=1 \ > > git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=master;destsuffix=${KMETA}" > > > > # Set default SRCREVs. Both the machine and meta SRCREVs are statically set > > I'm afraid this looks to be a bit of a horrible workaround/hack. It happens that > if you specify a branch and set nobranch it might do what you want but that > certainly isn't by design. > > Either the revision is in the branch or it isn't. The error is telling you the > configuration you set isn't valid and you really need to set a valid > configuration, i.e. a revision and a branch or a revision and set nobranch but > not both. > > I'm not sure I understand why the kernel would be rebasing after each kernel > release? Is this because it is one of the unversioned branches? Yah, it is exactly that. The -dev kernel has always been a rebase tree, like linux-next upstream. Since the BSP work started against it (the -dev tree), when I move on from a -dev version, I've been saving all work into versioned branches ... versus removing them (and storing the history in the kernel-cache). That being said, we merged some code some time ago that allows the -dev kernel to automatically switch to the versioned branches, versus the rebased "standard/*" branches. (that supports existing releases with the -dev kernel as I move the one in master to new versions). Have a look at do_validate_branches(), but unfortunately the fetcher error will have already been thrown and we can't adjust to the fixed SRCREV. The issue here is the attempt to pin the revision (like Richard is saying), since if you use AUTOREV the code I just mentioned code kicks in to make sure a versioned branch is used. Nothing should be released against -dev, so any issues with pinned SRCREVs and branches should be transient. I've floated the idea a few times that now that versioned branches are being created (to keep older dev kernels around) I could just create the branches from the start as versioned, and then you wouldn't have the issue you are seeing, even with a pinned SRCREV. We are in a middle state, since the structure of -dev has changed over time as more use for it appeared (and it was also created when even the linux-yocto branches were not versioned). But that isn't something that I could do until the next -dev version and only part of the next yocto release. But for the short term, the fetcher error you are seeing should happen once, and you could have a bbappend to adjust the KBRANCH accordingly, or similar .. which shouldn't be much extra effort, as you mentioned it was something in a release. If you have somewhere that you are setting the SRCREV, why not set the KBRANCH in the same place ? Cheers, Bruce > > I can fix the fetcher to hard error if both are set... > > Cheers, > > Richard > > > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
On Wed, Dec 01, 2021 at 04:27:44PM -0500, Bruce Ashfield wrote: > [Please note: This e-mail is from an EXTERNAL e-mail address] > > On Wed, Dec 1, 2021 at 6:28 AM Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > > > On Wed, 2021-12-01 at 12:39 +0800, Kevin Hao wrote: > > > By default the -dev kernel uses the "AUTOREV" to pull in the branch > > > head as the revision. Some of our BSPs are based on the -dev kernel and > > > we choose to nail down the kernel to a specific revision when releasing > > > our product by using some setting like below: > > > PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev" > > > SRCREV_machine:pn-linux-yocto-dev = "6fb48ae18a10770702266dd1f1aa500149e361ec" > > > KBRANCH:pn-linux-yocto-dev = "standard/x86" > > > LINUX_VERSION = "5.15" > > > > > > Since all the standard/* branches will be rebased after each kernel > > > version bump, we would get bitbake fetch failure due to that specific > > > commit is not reachable in the new version branch. This kind of issue > > > can be fixed by setting the "nobranch" parameter in the SRC_URI because > > > it will cause the fetcher to skip the SHA validation for the branch. > > > And this also won't cause other side effect because all the branches > > > will be created in the do_kernel_checkout() and the current branch will > > > be reset to the reversion we want in do_validate_branches(). > > > > > > Signed-off-by: Kevin Hao <kexin.hao@windriver.com> > > > --- > > > meta/recipes-kernel/linux/linux-yocto-dev.bb | 3 ++- > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb > > > index 6b6ea9a7e864..7204c3eddc11 100644 > > > --- a/meta/recipes-kernel/linux/linux-yocto-dev.bb > > > +++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb > > > @@ -19,7 +19,8 @@ include recipes-kernel/linux/linux-yocto-dev-revisions.inc > > > KBRANCH = "standard/base" > > > KMETA = "kernel-meta" > > > > > > -SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine \ > > > +# Set nobranch to skip the SHA validation for branch if a fixed revesion is used > > > +SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine;nobranch=1 \ > > > git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=master;destsuffix=${KMETA}" > > > > > > # Set default SRCREVs. Both the machine and meta SRCREVs are statically set > > > > I'm afraid this looks to be a bit of a horrible workaround/hack. It happens that > > if you specify a branch and set nobranch it might do what you want but that > > certainly isn't by design. > > > > Either the revision is in the branch or it isn't. The error is telling you the > > configuration you set isn't valid and you really need to set a valid > > configuration, i.e. a revision and a branch or a revision and set nobranch but > > not both. > > > > I'm not sure I understand why the kernel would be rebasing after each kernel > > release? Is this because it is one of the unversioned branches? > > Yah, it is exactly that. The -dev kernel has always been a rebase > tree, like linux-next upstream. > > Since the BSP work started against it (the -dev tree), when I move on > from a -dev version, I've been saving all work into versioned branches > ... versus removing them (and storing the history in the > kernel-cache). > > That being said, we merged some code some time ago that allows the > -dev kernel to automatically switch to the versioned branches, versus > the rebased "standard/*" branches. (that supports existing releases > with the -dev kernel as I move the one in master to new versions). > Have a look at do_validate_branches(), but unfortunately the fetcher > error will have already been thrown and we can't adjust to the fixed > SRCREV. > > The issue here is the attempt to pin the revision (like Richard is > saying), since if you use AUTOREV the code I just mentioned code kicks > in to make sure a versioned branch is used. Nothing should be released > against -dev, so any issues with pinned SRCREVs and branches should be > transient. > > I've floated the idea a few times that now that versioned branches are > being created (to keep older dev kernels around) I could just create > the branches from the start as versioned, and then you wouldn't have > the issue you are seeing, even with a pinned SRCREV. That would be great if you can do so. > We are in a > middle state, since the structure of -dev has changed over time as > more use for it appeared (and it was also created when even the > linux-yocto branches were not versioned). But that isn't something > that I could do until the next -dev version and only part of the next > yocto release. > > But for the short term, the fetcher error you are seeing should happen > once, and you could have a bbappend to adjust the KBRANCH accordingly, > or similar .. which shouldn't be much extra effort, as you mentioned > it was something in a release. If you have somewhere that you are > setting the SRCREV, why not set the KBRANCH in the same place ? Yes, it is pretty easy to fix this kind of issue in our development phase. It only become a big issue after the product is delivered to customer. Thanks, Kevin > > Cheers, > > Bruce > > > > > I can fix the fetcher to hard error if both are set... > > > > Cheers, > > > > Richard > > > > > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II
On Thu, Dec 2, 2021 at 2:23 AM Kevin Hao <kexin.hao@windriver.com> wrote: > > On Wed, Dec 01, 2021 at 04:27:44PM -0500, Bruce Ashfield wrote: > > [Please note: This e-mail is from an EXTERNAL e-mail address] > > > > On Wed, Dec 1, 2021 at 6:28 AM Richard Purdie > > <richard.purdie@linuxfoundation.org> wrote: > > > > > > On Wed, 2021-12-01 at 12:39 +0800, Kevin Hao wrote: > > > > By default the -dev kernel uses the "AUTOREV" to pull in the branch > > > > head as the revision. Some of our BSPs are based on the -dev kernel and > > > > we choose to nail down the kernel to a specific revision when releasing > > > > our product by using some setting like below: > > > > PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev" > > > > SRCREV_machine:pn-linux-yocto-dev = "6fb48ae18a10770702266dd1f1aa500149e361ec" > > > > KBRANCH:pn-linux-yocto-dev = "standard/x86" > > > > LINUX_VERSION = "5.15" > > > > > > > > Since all the standard/* branches will be rebased after each kernel > > > > version bump, we would get bitbake fetch failure due to that specific > > > > commit is not reachable in the new version branch. This kind of issue > > > > can be fixed by setting the "nobranch" parameter in the SRC_URI because > > > > it will cause the fetcher to skip the SHA validation for the branch. > > > > And this also won't cause other side effect because all the branches > > > > will be created in the do_kernel_checkout() and the current branch will > > > > be reset to the reversion we want in do_validate_branches(). > > > > > > > > Signed-off-by: Kevin Hao <kexin.hao@windriver.com> > > > > --- > > > > meta/recipes-kernel/linux/linux-yocto-dev.bb | 3 ++- > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb > > > > index 6b6ea9a7e864..7204c3eddc11 100644 > > > > --- a/meta/recipes-kernel/linux/linux-yocto-dev.bb > > > > +++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb > > > > @@ -19,7 +19,8 @@ include recipes-kernel/linux/linux-yocto-dev-revisions.inc > > > > KBRANCH = "standard/base" > > > > KMETA = "kernel-meta" > > > > > > > > -SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine \ > > > > +# Set nobranch to skip the SHA validation for branch if a fixed revesion is used > > > > +SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine;nobranch=1 \ > > > > git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=master;destsuffix=${KMETA}" > > > > > > > > # Set default SRCREVs. Both the machine and meta SRCREVs are statically set > > > > > > I'm afraid this looks to be a bit of a horrible workaround/hack. It happens that > > > if you specify a branch and set nobranch it might do what you want but that > > > certainly isn't by design. > > > > > > Either the revision is in the branch or it isn't. The error is telling you the > > > configuration you set isn't valid and you really need to set a valid > > > configuration, i.e. a revision and a branch or a revision and set nobranch but > > > not both. > > > > > > I'm not sure I understand why the kernel would be rebasing after each kernel > > > release? Is this because it is one of the unversioned branches? > > > > Yah, it is exactly that. The -dev kernel has always been a rebase > > tree, like linux-next upstream. > > > > Since the BSP work started against it (the -dev tree), when I move on > > from a -dev version, I've been saving all work into versioned branches > > ... versus removing them (and storing the history in the > > kernel-cache). > > > > That being said, we merged some code some time ago that allows the > > -dev kernel to automatically switch to the versioned branches, versus > > the rebased "standard/*" branches. (that supports existing releases > > with the -dev kernel as I move the one in master to new versions). > > Have a look at do_validate_branches(), but unfortunately the fetcher > > error will have already been thrown and we can't adjust to the fixed > > SRCREV. > > > > The issue here is the attempt to pin the revision (like Richard is > > saying), since if you use AUTOREV the code I just mentioned code kicks > > in to make sure a versioned branch is used. Nothing should be released > > against -dev, so any issues with pinned SRCREVs and branches should be > > transient. > > > > I've floated the idea a few times that now that versioned branches are > > being created (to keep older dev kernels around) I could just create > > the branches from the start as versioned, and then you wouldn't have > > the issue you are seeing, even with a pinned SRCREV. > > That would be great if you can do so. ok. Let's solve this with an explicit KBRANCH with your pinned SRCREVs for now, and I'll update -dev to use versioned branches for the current 5.16 cycle and all new versions that follow. Bruce > > > We are in a > > middle state, since the structure of -dev has changed over time as > > more use for it appeared (and it was also created when even the > > linux-yocto branches were not versioned). But that isn't something > > that I could do until the next -dev version and only part of the next > > yocto release. > > > > But for the short term, the fetcher error you are seeing should happen > > once, and you could have a bbappend to adjust the KBRANCH accordingly, > > or similar .. which shouldn't be much extra effort, as you mentioned > > it was something in a release. If you have somewhere that you are > > setting the SRCREV, why not set the KBRANCH in the same place ? > > Yes, it is pretty easy to fix this kind of issue in our development phase. > It only become a big issue after the product is delivered to customer. > > Thanks, > Kevin > > > > > Cheers, > > > > Bruce > > > > > > > > I can fix the fetcher to hard error if both are set... > > > > > > Cheers, > > > > > > Richard > > > > > > > > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II
On Thu, Dec 02, 2021 at 07:31:52AM -0500, Bruce Ashfield wrote: > > That would be great if you can do so. > > ok. Let's solve this with an explicit KBRANCH with your pinned SRCREVs > for now, and I'll update -dev to use versioned branches for the current > 5.16 cycle and all new versions that follow. Thanks Bruce. Kevin
diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb index 6b6ea9a7e864..7204c3eddc11 100644 --- a/meta/recipes-kernel/linux/linux-yocto-dev.bb +++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb @@ -19,7 +19,8 @@ include recipes-kernel/linux/linux-yocto-dev-revisions.inc KBRANCH = "standard/base" KMETA = "kernel-meta" -SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine \ +# Set nobranch to skip the SHA validation for branch if a fixed revesion is used +SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine;nobranch=1 \ git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=master;destsuffix=${KMETA}" # Set default SRCREVs. Both the machine and meta SRCREVs are statically set
By default the -dev kernel uses the "AUTOREV" to pull in the branch head as the revision. Some of our BSPs are based on the -dev kernel and we choose to nail down the kernel to a specific revision when releasing our product by using some setting like below: PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev" SRCREV_machine:pn-linux-yocto-dev = "6fb48ae18a10770702266dd1f1aa500149e361ec" KBRANCH:pn-linux-yocto-dev = "standard/x86" LINUX_VERSION = "5.15" Since all the standard/* branches will be rebased after each kernel version bump, we would get bitbake fetch failure due to that specific commit is not reachable in the new version branch. This kind of issue can be fixed by setting the "nobranch" parameter in the SRC_URI because it will cause the fetcher to skip the SHA validation for the branch. And this also won't cause other side effect because all the branches will be created in the do_kernel_checkout() and the current branch will be reset to the reversion we want in do_validate_branches(). Signed-off-by: Kevin Hao <kexin.hao@windriver.com> --- meta/recipes-kernel/linux/linux-yocto-dev.bb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)