| Message ID | 20240705113129.1798725-1-jstephan@baylibre.com |
|---|---|
| State | New |
| Headers | show |
| Series | kernel: fix do_symlink_kernsrc function | expand |
Julien Stephan via lists.openembedded.org <jstephan= baylibre.com@lists.openembedded.org> escreveu (sexta, 5/07/2024 à(s) 12:31): > According to the comment in do_symlink_kernsrc, this function exists > for compatibility with old style kernel recipes: > > # Old style kernels may set ${S} = ${WORKDIR}/git for example > > For such recipes S will always be different from STAGING_KERNEL_DIR. > This is fine for the first build or when unpack is rerun because new > sources are in S. However the following command breaks the build: > > bitbake -C do_symlink_kernsrc virtual/kernel > > At this point, S is a symlink to STAGING_KERNEL_DIR, (meaning S != > STAGING_KERNEL_DIR). We first remove the contents of STAGING_KERNEL_DIR > without removing the folder itself causing us to lose kernel sources. > Then we create a symlink from S to STAGING_KERNEL_DIR which results in the > following broken symlinks: > > ${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source > kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-source > > The build fails with the following error: > > ERROR: <linux_recipe> do_kernel_checkout: FileExistsError(17, 'File > exists') > ERROR: Task (<linux_recipe>:do_kernel_checkout) > failed with exit code '1' > > Attempting to access the kernel-source directory results in: > > ls: cannot access 'kernel-source': Too many levels of symbolic links > > Fix this by checking if S is a symlink, and if so, verifying whether the > symlink > points to STAGING_KERNEL_DIR to avoid losing sources > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > --- > meta/classes-recipe/kernel.bbclass | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/meta/classes-recipe/kernel.bbclass > b/meta/classes-recipe/kernel.bbclass > index 89badd90f18..e328151cb59 100644 > --- a/meta/classes-recipe/kernel.bbclass > +++ b/meta/classes-recipe/kernel.bbclass > @@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} > ${B} ${STAGING_KERNEL_BUILDD > python do_symlink_kernsrc () { > s = d.getVar("S") > kernsrc = d.getVar("STAGING_KERNEL_DIR") > - if s != kernsrc: > + if os.path.islink(s): > + _s = os.readlink(s) > + else: > + _s = s > + > + if _s != kernsrc: > bb.utils.mkdirhier(kernsrc) > bb.utils.remove(kernsrc, recurse=True) > if s[-1] == '/': > -- > 2.45.1 > What kernel bbclass are you using to build your kernel? Please note that a very similar logic was replicated on the kernel-yocto.bbclass do_kernel_checkout function so it will be good to test with and without the kernel-yocto.bbclass. https://git.yoctoproject.org/poky/tree/meta/classes-recipe/kernel-yocto.bbclass#n376 I recently came across this problem in a change that I ended up having to revert. https://github.com/Freescale/meta-freescale/pull/1843 https://github.com/Freescale/meta-freescale/pull/1857 Jose > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#201601): > https://lists.openembedded.org/g/openembedded-core/message/201601 > Mute This Topic: https://lists.openembedded.org/mt/107052142/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Hi Julien, did you see my previous patch and the follow-up discussion? Ticket: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15325 Previous discussion: https://lists.openembedded.org/g/openembedded-core/topic/103308574#msg195238 I think Richard wanted a solution where this errors out: > I'm very tempted to say the "using a non default S value" just error > out. > > Whilst I understand how we got to this level of complexity, I think it > will just end up causing us pain in the long run and I'd rather just > not support it. > > Cheers, > Richard Étienne On Fri, Jul 5, 2024 at 1:34 PM Julien Stephan via lists.openembedded.org <jstephan=baylibre.com@lists.openembedded.org> wrote: > According to the comment in do_symlink_kernsrc, this function exists > for compatibility with old style kernel recipes: > > # Old style kernels may set ${S} = ${WORKDIR}/git for example > > For such recipes S will always be different from STAGING_KERNEL_DIR. > This is fine for the first build or when unpack is rerun because new > sources are in S. However the following command breaks the build: > > bitbake -C do_symlink_kernsrc virtual/kernel > > At this point, S is a symlink to STAGING_KERNEL_DIR, (meaning S != > STAGING_KERNEL_DIR). We first remove the contents of STAGING_KERNEL_DIR > without removing the folder itself causing us to lose kernel sources. > Then we create a symlink from S to STAGING_KERNEL_DIR which results in the > following broken symlinks: > > ${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source > kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-source > > The build fails with the following error: > > ERROR: <linux_recipe> do_kernel_checkout: FileExistsError(17, 'File > exists') > ERROR: Task (<linux_recipe>:do_kernel_checkout) > failed with exit code '1' > > Attempting to access the kernel-source directory results in: > > ls: cannot access 'kernel-source': Too many levels of symbolic links > > Fix this by checking if S is a symlink, and if so, verifying whether the > symlink > points to STAGING_KERNEL_DIR to avoid losing sources > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > --- > meta/classes-recipe/kernel.bbclass | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/meta/classes-recipe/kernel.bbclass > b/meta/classes-recipe/kernel.bbclass > index 89badd90f18..e328151cb59 100644 > --- a/meta/classes-recipe/kernel.bbclass > +++ b/meta/classes-recipe/kernel.bbclass > @@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} > ${B} ${STAGING_KERNEL_BUILDD > python do_symlink_kernsrc () { > s = d.getVar("S") > kernsrc = d.getVar("STAGING_KERNEL_DIR") > - if s != kernsrc: > + if os.path.islink(s): > + _s = os.readlink(s) > + else: > + _s = s > + > + if _s != kernsrc: > bb.utils.mkdirhier(kernsrc) > bb.utils.remove(kernsrc, recurse=True) > if s[-1] == '/': > -- > 2.45.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#201601): > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_message_201601&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=N3u4aN5Ob1LBtzYAPrUgW7w1Yk_vN3KvuImWnp5g6LA&e= > Mute This Topic: > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_mt_107052142_7048771&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=VbM00DcAJR0ZUsuhhPWtLEKKZ5MiBkhgS_Q4Aq3X3KM&e= > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_unsub&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=7EbQmI1Lz7r_bu1RArY3qabfEim4bh48mxJPm1oN_N4&e= > [ecordonnier@snap.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Le ven. 5 juil. 2024 à 13:55, Jose Quaresma <quaresma.jose@gmail.com> a écrit : > > > > Julien Stephan via lists.openembedded.org <jstephan=baylibre.com@lists.openembedded.org> escreveu (sexta, 5/07/2024 à(s) 12:31): >> >> According to the comment in do_symlink_kernsrc, this function exists >> for compatibility with old style kernel recipes: >> >> # Old style kernels may set ${S} = ${WORKDIR}/git for example >> >> For such recipes S will always be different from STAGING_KERNEL_DIR. >> This is fine for the first build or when unpack is rerun because new >> sources are in S. However the following command breaks the build: >> >> bitbake -C do_symlink_kernsrc virtual/kernel >> >> At this point, S is a symlink to STAGING_KERNEL_DIR, (meaning S != >> STAGING_KERNEL_DIR). We first remove the contents of STAGING_KERNEL_DIR >> without removing the folder itself causing us to lose kernel sources. >> Then we create a symlink from S to STAGING_KERNEL_DIR which results in the >> following broken symlinks: >> >> ${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source >> kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-source >> >> The build fails with the following error: >> >> ERROR: <linux_recipe> do_kernel_checkout: FileExistsError(17, 'File exists') >> ERROR: Task (<linux_recipe>:do_kernel_checkout) >> failed with exit code '1' >> >> Attempting to access the kernel-source directory results in: >> >> ls: cannot access 'kernel-source': Too many levels of symbolic links >> >> Fix this by checking if S is a symlink, and if so, verifying whether the symlink >> points to STAGING_KERNEL_DIR to avoid losing sources >> >> Signed-off-by: Julien Stephan <jstephan@baylibre.com> >> --- >> meta/classes-recipe/kernel.bbclass | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass >> index 89badd90f18..e328151cb59 100644 >> --- a/meta/classes-recipe/kernel.bbclass >> +++ b/meta/classes-recipe/kernel.bbclass >> @@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} ${STAGING_KERNEL_BUILDD >> python do_symlink_kernsrc () { >> s = d.getVar("S") >> kernsrc = d.getVar("STAGING_KERNEL_DIR") >> - if s != kernsrc: >> + if os.path.islink(s): >> + _s = os.readlink(s) >> + else: >> + _s = s >> + >> + if _s != kernsrc: >> bb.utils.mkdirhier(kernsrc) >> bb.utils.remove(kernsrc, recurse=True) >> if s[-1] == '/': >> -- >> 2.45.1 > > > What kernel bbclass are you using to build your kernel? > Please note that a very similar logic was replicated on the kernel-yocto.bbclass do_kernel_checkout function > so it will be good to test with and without the kernel-yocto.bbclass. > https://git.yoctoproject.org/poky/tree/meta/classes-recipe/kernel-yocto.bbclass#n376 > > I recently came across this problem in a change that I ended up having to revert. > https://github.com/Freescale/meta-freescale/pull/1843 > https://github.com/Freescale/meta-freescale/pull/1857 > > Jose Interesting, I'll check this. Thank you for reporting :) Julien > >> >> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#201601): https://lists.openembedded.org/g/openembedded-core/message/201601 >> Mute This Topic: https://lists.openembedded.org/mt/107052142/5052612 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [quaresma.jose@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> > > > -- > Best regards, > > José Quaresma
Le ven. 5 juil. 2024 à 13:58, Etienne Cordonnier <ecordonnier@snap.com> a écrit : > > Hi Julien, > did you see my previous patch and the follow-up discussion? > Ticket: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15325 > Previous discussion: https://lists.openembedded.org/g/openembedded-core/topic/103308574#msg195238 > > I think Richard wanted a solution where this errors out: > > I'm very tempted to say the "using a non default S value" just error > > out. > > > > Whilst I understand how we got to this level of complexity, I think it > > will just end up causing us pain in the long run and I'd rather just > > not support it. > > > > Cheers, > > Richard > > Étienne > I didn't see it. Let me check this thread :) Cheers Julien > On Fri, Jul 5, 2024 at 1:34 PM Julien Stephan via lists.openembedded.org <jstephan=baylibre.com@lists.openembedded.org> wrote: >> >> According to the comment in do_symlink_kernsrc, this function exists >> for compatibility with old style kernel recipes: >> >> # Old style kernels may set ${S} = ${WORKDIR}/git for example >> >> For such recipes S will always be different from STAGING_KERNEL_DIR. >> This is fine for the first build or when unpack is rerun because new >> sources are in S. However the following command breaks the build: >> >> bitbake -C do_symlink_kernsrc virtual/kernel >> >> At this point, S is a symlink to STAGING_KERNEL_DIR, (meaning S != >> STAGING_KERNEL_DIR). We first remove the contents of STAGING_KERNEL_DIR >> without removing the folder itself causing us to lose kernel sources. >> Then we create a symlink from S to STAGING_KERNEL_DIR which results in the >> following broken symlinks: >> >> ${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source >> kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-source >> >> The build fails with the following error: >> >> ERROR: <linux_recipe> do_kernel_checkout: FileExistsError(17, 'File exists') >> ERROR: Task (<linux_recipe>:do_kernel_checkout) >> failed with exit code '1' >> >> Attempting to access the kernel-source directory results in: >> >> ls: cannot access 'kernel-source': Too many levels of symbolic links >> >> Fix this by checking if S is a symlink, and if so, verifying whether the symlink >> points to STAGING_KERNEL_DIR to avoid losing sources >> >> Signed-off-by: Julien Stephan <jstephan@baylibre.com> >> --- >> meta/classes-recipe/kernel.bbclass | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass >> index 89badd90f18..e328151cb59 100644 >> --- a/meta/classes-recipe/kernel.bbclass >> +++ b/meta/classes-recipe/kernel.bbclass >> @@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} ${STAGING_KERNEL_BUILDD >> python do_symlink_kernsrc () { >> s = d.getVar("S") >> kernsrc = d.getVar("STAGING_KERNEL_DIR") >> - if s != kernsrc: >> + if os.path.islink(s): >> + _s = os.readlink(s) >> + else: >> + _s = s >> + >> + if _s != kernsrc: >> bb.utils.mkdirhier(kernsrc) >> bb.utils.remove(kernsrc, recurse=True) >> if s[-1] == '/': >> -- >> 2.45.1 >> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#201601): https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_message_201601&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=N3u4aN5Ob1LBtzYAPrUgW7w1Yk_vN3KvuImWnp5g6LA&e= >> Mute This Topic: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_mt_107052142_7048771&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=VbM00DcAJR0ZUsuhhPWtLEKKZ5MiBkhgS_Q4Aq3X3KM&e= >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_unsub&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=7EbQmI1Lz7r_bu1RArY3qabfEim4bh48mxJPm1oN_N4&e= [ecordonnier@snap.com] >> -=-=-=-=-=-=-=-=-=-=-=- >>
diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass index 89badd90f18..e328151cb59 100644 --- a/meta/classes-recipe/kernel.bbclass +++ b/meta/classes-recipe/kernel.bbclass @@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} ${STAGING_KERNEL_BUILDD python do_symlink_kernsrc () { s = d.getVar("S") kernsrc = d.getVar("STAGING_KERNEL_DIR") - if s != kernsrc: + if os.path.islink(s): + _s = os.readlink(s) + else: + _s = s + + if _s != kernsrc: bb.utils.mkdirhier(kernsrc) bb.utils.remove(kernsrc, recurse=True) if s[-1] == '/':
According to the comment in do_symlink_kernsrc, this function exists for compatibility with old style kernel recipes: # Old style kernels may set ${S} = ${WORKDIR}/git for example For such recipes S will always be different from STAGING_KERNEL_DIR. This is fine for the first build or when unpack is rerun because new sources are in S. However the following command breaks the build: bitbake -C do_symlink_kernsrc virtual/kernel At this point, S is a symlink to STAGING_KERNEL_DIR, (meaning S != STAGING_KERNEL_DIR). We first remove the contents of STAGING_KERNEL_DIR without removing the folder itself causing us to lose kernel sources. Then we create a symlink from S to STAGING_KERNEL_DIR which results in the following broken symlinks: ${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-source The build fails with the following error: ERROR: <linux_recipe> do_kernel_checkout: FileExistsError(17, 'File exists') ERROR: Task (<linux_recipe>:do_kernel_checkout) failed with exit code '1' Attempting to access the kernel-source directory results in: ls: cannot access 'kernel-source': Too many levels of symbolic links Fix this by checking if S is a symlink, and if so, verifying whether the symlink points to STAGING_KERNEL_DIR to avoid losing sources Signed-off-by: Julien Stephan <jstephan@baylibre.com> --- meta/classes-recipe/kernel.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)