| Message ID | 20230416103052.28268-1-christoph.lauer@email.de |
|---|---|
| State | New |
| Headers | show |
| Series | make-mod-scripts: preserve libraries when rm_work is used | expand |
Hi Christoph, This patch is also applicable on kirstone so it can be backported. I use in my distro RM_WORK_EXCLUDE += "make-mod-scripts" but your fix looks better. Thanks. Jose Christoph Lauer <christoph.lauer@email.de> escreveu no dia domingo, 16/04/2023 à(s) 11:31: > From: Christoph Lauer <christoph.lauer@xtronic.de> > > With rm_work active, external module signing throws an error: > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: > cannot open shared object file: No such file or directory > Preserve libraries that sign-file script needs during runtime. > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > --- > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > index 28e0807d1d..0e24efc597 100644 > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > @@ -32,3 +32,6 @@ do_configure() { > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > done > } > + > +# keep native libraries required for module signing > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > -- > 2.17.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180113): > https://lists.openembedded.org/g/openembedded-core/message/180113 > Mute This Topic: https://lists.openembedded.org/mt/98296212/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Hi Jose, Thanks for your comment; I intend to backport the patch to all actively maintained branches once it is accepted for master merge. Christoph Am 17.04.23 um 12:19 schrieb Jose Quaresma: > Hi Christoph, > > This patch is also applicable on kirstone so it can be backported. > I use in my distro RM_WORK_EXCLUDE += "make-mod-scripts" but your fix > looks better. > Thanks. > > Jose > > Christoph Lauer <christoph.lauer@email.de > <mailto:christoph.lauer@email.de>> escreveu no dia domingo, 16/04/2023 > à(s) 11:31: > > From: Christoph Lauer <christoph.lauer@xtronic.de > <mailto:christoph.lauer@xtronic.de>> > > With rm_work active, external module signing throws an error: > scripts/sign-file: error while loading shared libraries: > libcrypto.so.3: cannot open shared object file: No such file or > directory > Preserve libraries that sign-file script needs during runtime. > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de > <mailto:christoph.lauer@xtronic.de>> > --- > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > <http://make-mod-scripts_1.0.bb> | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git > a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > <http://make-mod-scripts_1.0.bb> > b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > <http://make-mod-scripts_1.0.bb> > index 28e0807d1d..0e24efc597 100644 > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > <http://make-mod-scripts_1.0.bb> > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > <http://make-mod-scripts_1.0.bb> > @@ -32,3 +32,6 @@ do_configure() { > -C ${STAGING_KERNEL_DIR} > O=${STAGING_KERNEL_BUILDDIR} $t > done > } > + > +# keep native libraries required for module signing > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > -- > 2.17.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180113): > https://lists.openembedded.org/g/openembedded-core/message/180113 > <https://lists.openembedded.org/g/openembedded-core/message/180113> > Mute This Topic: https://lists.openembedded.org/mt/98296212/5052612 > <https://lists.openembedded.org/mt/98296212/5052612> > Group Owner: openembedded-core+owner@lists.openembedded.org > <mailto:openembedded-core%2Bowner@lists.openembedded.org> > Unsubscribe: > https://lists.openembedded.org/g/openembedded-core/unsub > <https://lists.openembedded.org/g/openembedded-core/unsub> > [quaresma.jose@gmail.com <mailto:quaresma.jose@gmail.com>] > -=-=-=-=-=-=-=-=-=-=-=- > > > > -- > Best regards, > > José Quaresma
On Sun, Apr 16, 2023 at 6:31 AM Christoph Lauer <christoph.lauer@email.de> wrote: > > From: Christoph Lauer <christoph.lauer@xtronic.de> > > With rm_work active, external module signing throws an error: > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: cannot open shared object file: No such file or directory > Preserve libraries that sign-file script needs during runtime. > We are by design installing the output of the scripts target to the staging kernel build dir. If there's some part of the build and install that isn't going to the shared location, then we should be making sure it goes to that shared location (and is cleaned with the rest of the artifacts). Users of those same scripts need to be able to locate and load the artifacts from the staging kernel build dir, but that's solvable/preferable. If you've tried the above and it doesn't work, it would be useful to capture that in the commit log. Bruce > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > --- > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > index 28e0807d1d..0e24efc597 100644 > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > @@ -32,3 +32,6 @@ do_configure() { > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > done > } > + > +# keep native libraries required for module signing > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > -- > 2.17.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180113): https://lists.openembedded.org/g/openembedded-core/message/180113 > Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: > From: Christoph Lauer <christoph.lauer@xtronic.de> > > With rm_work active, external module signing throws an error: > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: cannot open shared object file: No such file or directory > Preserve libraries that sign-file script needs during runtime. > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > --- > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > index 28e0807d1d..0e24efc597 100644 > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > @@ -32,3 +32,6 @@ do_configure() { > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > done > } > + > +# keep native libraries required for module signing > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" I'm really reluctant to take this change as it isn't the way dependencies are meant to work. It sounds like something in a shared workdir is depending on something in a recipe workdir and we simply don't support that. Everything needed should be in the shared workdir. At best this is a bandaid and there will be other ways to make this fail such as cleaning make-mod-scripts. I'm even less keen to take it when I think it's going to be backported "everywhere" as if is the correct solution too. I don't know what the right fix is unfortunately. I'm sure people would like me to think about it and come up with one but there are simply too many different things people would like me to do that with and even for me, it does take a while to work these things out. I'm just out of bandwidth, sorry :( Cheers, Richard
Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia segunda, 17/04/2023 à(s) 20:51: > On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: > > From: Christoph Lauer <christoph.lauer@xtronic.de> > > > > With rm_work active, external module signing throws an error: > > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: > cannot open shared object file: No such file or directory > > Preserve libraries that sign-file script needs during runtime. > > > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > > --- > > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb > > index 28e0807d1d..0e24efc597 100644 > > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > > @@ -32,3 +32,6 @@ do_configure() { > > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > > done > > } > > + > > +# keep native libraries required for module signing > > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > > I'm really reluctant to take this change as it isn't the way > dependencies are meant to work. > > It sounds like something in a shared workdir is depending on something > in a recipe workdir and we simply don't support that. Everything needed > should be in the shared workdir. At best this is a bandaid and there > will be other ways to make this fail such as cleaning make-mod-scripts. > The problem is because for signing the kernel modules the sign-file.c [1] is linked dynamically with openssl-native. This works when building the in tree kernel modules but will fail when we try to sing any out of tree kernel module. To sign the out of tree kernel we will use the binaries from the shared workdir but the native libcrypto.so.3 is removed by the rm_work bbclass. We need to link the sign-file statically otherwise it will not work with the rm_work bbclass. [1] https://github.com/torvalds/linux/blob/master/scripts/sign-file.c Another solution for this problem can be changing the make-mod-scripts to be a native tool and in this way they will be installed and the dependencies will be handled correctly. > I'm even less keen to take it when I think it's going to be backported > "everywhere" as if is the correct solution too. > > I don't know what the right fix is unfortunately. I'm sure people would > like me to think about it and come up with one but there are simply too > many different things people would like me to do that with and even for > me, it does take a while to work these things out. I'm just out of > bandwidth, sorry :( > It is true that it is not the correct solution but it is the most suitable in my opinion. I totally understand what you say and I'm a little sorry that I could still help in this same fix. This problem is something I would also like to fix because I am using the RM_WORK_EXCLUDE for quite some time to fix this issue on my distro. I would like to convert the make-mod-scripts to be a native tool but I haven't had time for that either. Sorry and thank you for all your dedication and help. Jose > Cheers, > > Richard > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180167): > https://lists.openembedded.org/g/openembedded-core/message/180167 > Mute This Topic: https://lists.openembedded.org/mt/98296212/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Mon, Apr 17, 2023 at 6:31 PM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > > > Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia segunda, 17/04/2023 à(s) 20:51: >> >> On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: >> > From: Christoph Lauer <christoph.lauer@xtronic.de> >> > >> > With rm_work active, external module signing throws an error: >> > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: cannot open shared object file: No such file or directory >> > Preserve libraries that sign-file script needs during runtime. >> > >> > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> >> > --- >> > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ >> > 1 file changed, 3 insertions(+) >> > >> > diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb >> > index 28e0807d1d..0e24efc597 100644 >> > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb >> > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb >> > @@ -32,3 +32,6 @@ do_configure() { >> > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t >> > done >> > } >> > + >> > +# keep native libraries required for module signing >> > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" >> >> I'm really reluctant to take this change as it isn't the way >> dependencies are meant to work. >> >> It sounds like something in a shared workdir is depending on something >> in a recipe workdir and we simply don't support that. Everything needed >> should be in the shared workdir. At best this is a bandaid and there >> will be other ways to make this fail such as cleaning make-mod-scripts. > > > The problem is because for signing the kernel modules the sign-file.c [1] is linked dynamically with openssl-native. > This works when building the in tree kernel modules but will fail when we try to sing any out of tree kernel module. > To sign the out of tree kernel we will use the binaries from the shared workdir but the native libcrypto.so.3 is removed by > the rm_work bbclass. We need to link the sign-file statically otherwise it will not work with the rm_work bbclass. > > [1] https://github.com/torvalds/linux/blob/master/scripts/sign-file.c > > Another solution for this problem can be changing the make-mod-scripts to be a native tool and in this way > they will be installed and the dependencies will be handled correctly. There would very likely be different issues if the scripts were generated and then packaged as a native tool / package. Since they are so tightly coupled to the kernel. We'd just trade one set of issues for another (out of sync artifacts, etc). I'm going to hack on this a bit. That being said, I've never done any module signing .. since I don't need it in my development workflow. Is there a canonical guide to getting it setup so I can test my static link and relocated artifacts fixes ? is it with meta-integrity and the kernel-modsign bbclass ? Bruce Bruce > >> >> I'm even less keen to take it when I think it's going to be backported >> "everywhere" as if is the correct solution too. >> >> I don't know what the right fix is unfortunately. I'm sure people would >> like me to think about it and come up with one but there are simply too >> many different things people would like me to do that with and even for >> me, it does take a while to work these things out. I'm just out of >> bandwidth, sorry :( > > > It is true that it is not the correct solution but it is the most suitable in my opinion. > I totally understand what you say and I'm a little sorry that I could still help in this same fix. > > This problem is something I would also like to fix because I am using the RM_WORK_EXCLUDE > for quite some time to fix this issue on my distro. > I would like to convert the make-mod-scripts to be a native tool but I haven't had time for that either. > > Sorry and thank you for all your dedication and help. > > Jose > >> >> Cheers, >> >> Richard >> >> >> >> >> > > > -- > Best regards, > > José Quaresma > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180169): https://lists.openembedded.org/g/openembedded-core/message/180169 > Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
On Tue, Apr 18, 2023 at 4:25 PM Bruce Ashfield via lists.openembedded.org <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > > On Mon, Apr 17, 2023 at 6:31 PM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > > > > > > > Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia segunda, 17/04/2023 à(s) 20:51: > >> > >> On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: > >> > From: Christoph Lauer <christoph.lauer@xtronic.de> > >> > > >> > With rm_work active, external module signing throws an error: > >> > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: cannot open shared object file: No such file or directory > >> > Preserve libraries that sign-file script needs during runtime. > >> > > >> > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > >> > --- > >> > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ > >> > 1 file changed, 3 insertions(+) > >> > > >> > diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > >> > index 28e0807d1d..0e24efc597 100644 > >> > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > >> > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > >> > @@ -32,3 +32,6 @@ do_configure() { > >> > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > >> > done > >> > } > >> > + > >> > +# keep native libraries required for module signing > >> > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > >> > >> I'm really reluctant to take this change as it isn't the way > >> dependencies are meant to work. > >> > >> It sounds like something in a shared workdir is depending on something > >> in a recipe workdir and we simply don't support that. Everything needed > >> should be in the shared workdir. At best this is a bandaid and there > >> will be other ways to make this fail such as cleaning make-mod-scripts. > > > > > > The problem is because for signing the kernel modules the sign-file.c [1] is linked dynamically with openssl-native. > > This works when building the in tree kernel modules but will fail when we try to sing any out of tree kernel module. > > To sign the out of tree kernel we will use the binaries from the shared workdir but the native libcrypto.so.3 is removed by > > the rm_work bbclass. We need to link the sign-file statically otherwise it will not work with the rm_work bbclass. > > > > [1] https://github.com/torvalds/linux/blob/master/scripts/sign-file.c > > > > Another solution for this problem can be changing the make-mod-scripts to be a native tool and in this way > > they will be installed and the dependencies will be handled correctly. > > There would very likely be different issues if the scripts were > generated and then packaged as a native tool / package. Since they are > so tightly coupled to the kernel. We'd just trade one set of issues > for another (out of sync artifacts, etc). > > I'm going to hack on this a bit. > > That being said, I've never done any module signing .. since I don't > need it in my development workflow. > > Is there a canonical guide to getting it setup so I can test my static > link and relocated artifacts fixes ? is it with meta-integrity and the > kernel-modsign bbclass ? or are you maining just using the force-signing fragments (or equivalent) kernel configuration ? Bruce > > Bruce > > > Bruce > > > > >> > >> I'm even less keen to take it when I think it's going to be backported > >> "everywhere" as if is the correct solution too. > >> > >> I don't know what the right fix is unfortunately. I'm sure people would > >> like me to think about it and come up with one but there are simply too > >> many different things people would like me to do that with and even for > >> me, it does take a while to work these things out. I'm just out of > >> bandwidth, sorry :( > > > > > > It is true that it is not the correct solution but it is the most suitable in my opinion. > > I totally understand what you say and I'm a little sorry that I could still help in this same fix. > > > > This problem is something I would also like to fix because I am using the RM_WORK_EXCLUDE > > for quite some time to fix this issue on my distro. > > I would like to convert the make-mod-scripts to be a native tool but I haven't had time for that either. > > > > Sorry and thank you for all your dedication and help. > > > > Jose > > > >> > >> Cheers, > >> > >> Richard > >> > >> > >> > >> > >> > > > > > > -- > > Best regards, > > > > José Quaresma > > > > > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180200): https://lists.openembedded.org/g/openembedded-core/message/180200 > Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Tue, 2023-04-18 at 16:25 -0400, Bruce Ashfield wrote: > On Mon, Apr 17, 2023 at 6:31 PM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > > > > > > > Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia segunda, 17/04/2023 à(s) 20:51: > > > > > > On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: > > > > From: Christoph Lauer <christoph.lauer@xtronic.de> > > > > > > > > With rm_work active, external module signing throws an error: > > > > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: cannot open shared object file: No such file or directory > > > > Preserve libraries that sign-file script needs during runtime. > > > > > > > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > > > > --- > > > > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ > > > > 1 file changed, 3 insertions(+) > > > > > > > > diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > > > > index 28e0807d1d..0e24efc597 100644 > > > > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > > > > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > > > > @@ -32,3 +32,6 @@ do_configure() { > > > > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > > > > done > > > > } > > > > + > > > > +# keep native libraries required for module signing > > > > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > > > > > > I'm really reluctant to take this change as it isn't the way > > > dependencies are meant to work. > > > > > > It sounds like something in a shared workdir is depending on something > > > in a recipe workdir and we simply don't support that. Everything needed > > > should be in the shared workdir. At best this is a bandaid and there > > > will be other ways to make this fail such as cleaning make-mod-scripts. > > > > > > The problem is because for signing the kernel modules the sign-file.c [1] is linked dynamically with openssl-native. > > This works when building the in tree kernel modules but will fail when we try to sing any out of tree kernel module. > > To sign the out of tree kernel we will use the binaries from the shared workdir but the native libcrypto.so.3 is removed by > > the rm_work bbclass. We need to link the sign-file statically otherwise it will not work with the rm_work bbclass. > > > > [1] https://github.com/torvalds/linux/blob/master/scripts/sign-file.c > > > > Another solution for this problem can be changing the make-mod-scripts to be a native tool and in this way > > they will be installed and the dependencies will be handled correctly. > > There would very likely be different issues if the scripts were > generated and then packaged as a native tool / package. Since they are > so tightly coupled to the kernel. We'd just trade one set of issues > for another (out of sync artifacts, etc). > > I'm going to hack on this a bit. > > That being said, I've never done any module signing .. since I don't > need it in my development workflow. > > Is there a canonical guide to getting it setup so I can test my static > link and relocated artifacts fixes ? is it with meta-integrity and the > kernel-modsign bbclass ? I did think about this a bit more. It does likely depend on the version of libcrypto from the host system as to whether it reproduces or not. The possible solution ideas I came up with are: a) statically link sign-file so we don't need libcrypto b) copy the libcrypto.so into a known location in the shared workdir (probably some new path) and then adding an RPATH/RUNPATH using chrpath to the binary. Cheers, Richard
On Tue, Apr 18, 2023 at 5:04 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Tue, 2023-04-18 at 16:25 -0400, Bruce Ashfield wrote: > > On Mon, Apr 17, 2023 at 6:31 PM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > > > > > > > > > > > Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia segunda, 17/04/2023 à(s) 20:51: > > > > > > > > On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: > > > > > From: Christoph Lauer <christoph.lauer@xtronic.de> > > > > > > > > > > With rm_work active, external module signing throws an error: > > > > > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: cannot open shared object file: No such file or directory > > > > > Preserve libraries that sign-file script needs during runtime. > > > > > > > > > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > > > > > --- > > > > > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ > > > > > 1 file changed, 3 insertions(+) > > > > > > > > > > diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > > > > > index 28e0807d1d..0e24efc597 100644 > > > > > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > > > > > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > > > > > @@ -32,3 +32,6 @@ do_configure() { > > > > > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > > > > > done > > > > > } > > > > > + > > > > > +# keep native libraries required for module signing > > > > > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > > > > > > > > I'm really reluctant to take this change as it isn't the way > > > > dependencies are meant to work. > > > > > > > > It sounds like something in a shared workdir is depending on something > > > > in a recipe workdir and we simply don't support that. Everything needed > > > > should be in the shared workdir. At best this is a bandaid and there > > > > will be other ways to make this fail such as cleaning make-mod-scripts. > > > > > > > > > The problem is because for signing the kernel modules the sign-file.c [1] is linked dynamically with openssl-native. > > > This works when building the in tree kernel modules but will fail when we try to sing any out of tree kernel module. > > > To sign the out of tree kernel we will use the binaries from the shared workdir but the native libcrypto.so.3 is removed by > > > the rm_work bbclass. We need to link the sign-file statically otherwise it will not work with the rm_work bbclass. > > > > > > [1] https://github.com/torvalds/linux/blob/master/scripts/sign-file.c > > > > > > Another solution for this problem can be changing the make-mod-scripts to be a native tool and in this way > > > they will be installed and the dependencies will be handled correctly. > > > > There would very likely be different issues if the scripts were > > generated and then packaged as a native tool / package. Since they are > > so tightly coupled to the kernel. We'd just trade one set of issues > > for another (out of sync artifacts, etc). > > > > I'm going to hack on this a bit. > > > > That being said, I've never done any module signing .. since I don't > > need it in my development workflow. > > > > Is there a canonical guide to getting it setup so I can test my static > > link and relocated artifacts fixes ? is it with meta-integrity and the > > kernel-modsign bbclass ? > > I did think about this a bit more. It does likely depend on the version > of libcrypto from the host system as to whether it reproduces or not. > The possible solution ideas I came up with are: > > a) statically link sign-file so we don't need libcrypto > b) copy the libcrypto.so into a known location in the shared workdir > (probably some new path) and then adding an RPATH/RUNPATH using chrpath > to the binary. Agreed. I have sign-file statically building here, and it works, but objtool is blowing up under static linking. If I can't break the tools into separate bits, or fix that static link .. My other idea is the same as yours, if we copy out the .so and make sure it is in a recognized location in the artifacts dir, we are good to go. Bruce > > Cheers, > > Richard > >
Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, 18/04/2023 à(s) 22:12: > On Tue, Apr 18, 2023 at 5:04 PM Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > > > On Tue, 2023-04-18 at 16:25 -0400, Bruce Ashfield wrote: > > > On Mon, Apr 17, 2023 at 6:31 PM Jose Quaresma <quaresma.jose@gmail.com> > wrote: > > > > > > > > > > > > > > > > Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia > segunda, 17/04/2023 à(s) 20:51: > > > > > > > > > > On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: > > > > > > From: Christoph Lauer <christoph.lauer@xtronic.de> > > > > > > > > > > > > With rm_work active, external module signing throws an error: > > > > > > scripts/sign-file: error while loading shared libraries: > libcrypto.so.3: cannot open shared object file: No such file or directory > > > > > > Preserve libraries that sign-file script needs during runtime. > > > > > > > > > > > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > > > > > > --- > > > > > > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | > 3 +++ > > > > > > 1 file changed, 3 insertions(+) > > > > > > > > > > > > diff --git a/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb > > > > > > index 28e0807d1d..0e24efc597 100644 > > > > > > --- a/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb > > > > > > +++ b/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb > > > > > > @@ -32,3 +32,6 @@ do_configure() { > > > > > > -C ${STAGING_KERNEL_DIR} > O=${STAGING_KERNEL_BUILDDIR} $t > > > > > > done > > > > > > } > > > > > > + > > > > > > +# keep native libraries required for module signing > > > > > > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > > > > > > > > > > I'm really reluctant to take this change as it isn't the way > > > > > dependencies are meant to work. > > > > > > > > > > It sounds like something in a shared workdir is depending on > something > > > > > in a recipe workdir and we simply don't support that. Everything > needed > > > > > should be in the shared workdir. At best this is a bandaid and > there > > > > > will be other ways to make this fail such as cleaning > make-mod-scripts. > > > > > > > > > > > > The problem is because for signing the kernel modules the > sign-file.c [1] is linked dynamically with openssl-native. > > > > This works when building the in tree kernel modules but will fail > when we try to sing any out of tree kernel module. > > > > To sign the out of tree kernel we will use the binaries from the > shared workdir but the native libcrypto.so.3 is removed by > > > > the rm_work bbclass. We need to link the sign-file statically > otherwise it will not work with the rm_work bbclass. > > > > > > > > [1] > https://github.com/torvalds/linux/blob/master/scripts/sign-file.c > > > > > > > > Another solution for this problem can be changing the > make-mod-scripts to be a native tool and in this way > > > > they will be installed and the dependencies will be handled > correctly. > > > > > > There would very likely be different issues if the scripts were > > > generated and then packaged as a native tool / package. Since they are > > > so tightly coupled to the kernel. We'd just trade one set of issues > > > for another (out of sync artifacts, etc). > Maybe some new issues will come with this change but this will be more aligned with majority of the other tools. This is something I will keep in my TODO list. > > > > > > I'm going to hack on this a bit. > > > > > > That being said, I've never done any module signing .. since I don't > > > need it in my development workflow. > > > > > > Is there a canonical guide to getting it setup so I can test my static > > > link and relocated artifacts fixes ? is it with meta-integrity and the > > > kernel-modsign bbclass ? > Yes, I am using the kernel-modsign bbclass of meta-security. The step to needed is add modsign in DISTRO_FUTURES and provide the required keys setting the MODSIGN_* variable in the bbclass > > > I did think about this a bit more. It does likely depend on the version > > of libcrypto from the host system as to whether it reproduces or not. > > The possible solution ideas I came up with are: > > > > a) statically link sign-file so we don't need libcrypto > > b) copy the libcrypto.so into a known location in the shared workdir > > (probably some new path) and then adding an RPATH/RUNPATH using chrpath > > to the binary. > > Agreed. I have sign-file statically building here, and it works, but > objtool is blowing up under static linking. > Building the sign-file statically looks like a good solution but I have no idea of the side effects. > > If I can't break the tools into separate bits, or fix that static link > .. My other idea is the same as yours, if we copy out the .so and make > sure it is in a recognized location in the artifacts dir, we are good > to go. > But I believe that for copying it will require copying the full native sysroot dependency chain and not only libcrypto.so. Jose > > Bruce > > > > > Cheers, > > > > Richard > > > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II >
On Wed, Apr 19, 2023 at 9:52 AM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, 18/04/2023 à(s) 22:12: >> >> On Tue, Apr 18, 2023 at 5:04 PM Richard Purdie >> <richard.purdie@linuxfoundation.org> wrote: >> > >> > On Tue, 2023-04-18 at 16:25 -0400, Bruce Ashfield wrote: >> > > On Mon, Apr 17, 2023 at 6:31 PM Jose Quaresma <quaresma.jose@gmail.com> wrote: >> > > > >> > > > >> > > > >> > > > Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia segunda, 17/04/2023 à(s) 20:51: >> > > > > >> > > > > On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: >> > > > > > From: Christoph Lauer <christoph.lauer@xtronic.de> >> > > > > > >> > > > > > With rm_work active, external module signing throws an error: >> > > > > > scripts/sign-file: error while loading shared libraries: libcrypto.so.3: cannot open shared object file: No such file or directory >> > > > > > Preserve libraries that sign-file script needs during runtime. >> > > > > > >> > > > > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> >> > > > > > --- >> > > > > > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 3 +++ >> > > > > > 1 file changed, 3 insertions(+) >> > > > > > >> > > > > > diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb >> > > > > > index 28e0807d1d..0e24efc597 100644 >> > > > > > --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb >> > > > > > +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb >> > > > > > @@ -32,3 +32,6 @@ do_configure() { >> > > > > > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t >> > > > > > done >> > > > > > } >> > > > > > + >> > > > > > +# keep native libraries required for module signing >> > > > > > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" >> > > > > >> > > > > I'm really reluctant to take this change as it isn't the way >> > > > > dependencies are meant to work. >> > > > > >> > > > > It sounds like something in a shared workdir is depending on something >> > > > > in a recipe workdir and we simply don't support that. Everything needed >> > > > > should be in the shared workdir. At best this is a bandaid and there >> > > > > will be other ways to make this fail such as cleaning make-mod-scripts. >> > > > >> > > > >> > > > The problem is because for signing the kernel modules the sign-file.c [1] is linked dynamically with openssl-native. >> > > > This works when building the in tree kernel modules but will fail when we try to sing any out of tree kernel module. >> > > > To sign the out of tree kernel we will use the binaries from the shared workdir but the native libcrypto.so.3 is removed by >> > > > the rm_work bbclass. We need to link the sign-file statically otherwise it will not work with the rm_work bbclass. >> > > > >> > > > [1] https://github.com/torvalds/linux/blob/master/scripts/sign-file.c >> > > > >> > > > Another solution for this problem can be changing the make-mod-scripts to be a native tool and in this way >> > > > they will be installed and the dependencies will be handled correctly. >> > > >> > > There would very likely be different issues if the scripts were >> > > generated and then packaged as a native tool / package. Since they are >> > > so tightly coupled to the kernel. We'd just trade one set of issues >> > > for another (out of sync artifacts, etc). > > > Maybe some new issues will come with this change but this will be more aligned with > majority of the other tools. This is something I will keep in my TODO list. > >> >> > > >> > > I'm going to hack on this a bit. >> > > >> > > That being said, I've never done any module signing .. since I don't >> > > need it in my development workflow. >> > > >> > > Is there a canonical guide to getting it setup so I can test my static >> > > link and relocated artifacts fixes ? is it with meta-integrity and the >> > > kernel-modsign bbclass ? > > > Yes, I am using the kernel-modsign bbclass of meta-security. > The step to needed is add modsign in DISTRO_FUTURES and > provide the required keys setting the MODSIGN_* variable in the bbclass > >> > >> > I did think about this a bit more. It does likely depend on the version >> > of libcrypto from the host system as to whether it reproduces or not. >> > The possible solution ideas I came up with are: >> > >> > a) statically link sign-file so we don't need libcrypto >> > b) copy the libcrypto.so into a known location in the shared workdir >> > (probably some new path) and then adding an RPATH/RUNPATH using chrpath >> > to the binary. >> >> Agreed. I have sign-file statically building here, and it works, but >> objtool is blowing up under static linking. > > > Building the sign-file statically looks like a good solution but I have no idea of the side effects. There's no side effects to the tool itself (outside of the normal "the binary is a bit larger", etc, it is some of the other utilities that are causing issues. > >> >> >> If I can't break the tools into separate bits, or fix that static link >> .. My other idea is the same as yours, if we copy out the .so and make >> sure it is in a recognized location in the artifacts dir, we are good >> to go. > > > But I believe that for copying it will require copying the full native sysroot dependency chain and > not only libcrypto.so. The only thing missing is libcrypto (from my checking), so it doesn't look like we'd need any more than that. But yes, it would be expected we'd have to bring in all the requirements (but many of the common ones are already provided by the default native environment). Bruce > > Jose > >> >> >> Bruce >> >> > >> > Cheers, >> > >> > Richard >> > >> > >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II > > > > -- > Best regards, > > José Quaresma
Hi, Not related with the previous discussion but just for your information. The rm_work.bbclass has an exception for the kernel recipes [1]. So I don't understand why we can't do the same for the make-mod-scripts who is the twin brother of all these kernel recipes. [1] https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 Jose Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia quarta, 19/04/2023 à(s) 14:59: > On Wed, Apr 19, 2023 at 9:52 AM Jose Quaresma <quaresma.jose@gmail.com> > wrote: > > > > > > > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, > 18/04/2023 à(s) 22:12: > >> > >> On Tue, Apr 18, 2023 at 5:04 PM Richard Purdie > >> <richard.purdie@linuxfoundation.org> wrote: > >> > > >> > On Tue, 2023-04-18 at 16:25 -0400, Bruce Ashfield wrote: > >> > > On Mon, Apr 17, 2023 at 6:31 PM Jose Quaresma < > quaresma.jose@gmail.com> wrote: > >> > > > > >> > > > > >> > > > > >> > > > Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no > dia segunda, 17/04/2023 à(s) 20:51: > >> > > > > > >> > > > > On Sun, 2023-04-16 at 12:30 +0200, Christoph Lauer wrote: > >> > > > > > From: Christoph Lauer <christoph.lauer@xtronic.de> > >> > > > > > > >> > > > > > With rm_work active, external module signing throws an error: > >> > > > > > scripts/sign-file: error while loading shared libraries: > libcrypto.so.3: cannot open shared object file: No such file or directory > >> > > > > > Preserve libraries that sign-file script needs during runtime. > >> > > > > > > >> > > > > > Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> > >> > > > > > --- > >> > > > > > meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb > | 3 +++ > >> > > > > > 1 file changed, 3 insertions(+) > >> > > > > > > >> > > > > > diff --git a/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb > >> > > > > > index 28e0807d1d..0e24efc597 100644 > >> > > > > > --- a/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb > >> > > > > > +++ b/meta/recipes-kernel/make-mod-scripts/ > make-mod-scripts_1.0.bb > >> > > > > > @@ -32,3 +32,6 @@ do_configure() { > >> > > > > > -C ${STAGING_KERNEL_DIR} > O=${STAGING_KERNEL_BUILDDIR} $t > >> > > > > > done > >> > > > > > } > >> > > > > > + > >> > > > > > +# keep native libraries required for module signing > >> > > > > > +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" > >> > > > > > >> > > > > I'm really reluctant to take this change as it isn't the way > >> > > > > dependencies are meant to work. > >> > > > > > >> > > > > It sounds like something in a shared workdir is depending on > something > >> > > > > in a recipe workdir and we simply don't support that. > Everything needed > >> > > > > should be in the shared workdir. At best this is a bandaid and > there > >> > > > > will be other ways to make this fail such as cleaning > make-mod-scripts. > >> > > > > >> > > > > >> > > > The problem is because for signing the kernel modules the > sign-file.c [1] is linked dynamically with openssl-native. > >> > > > This works when building the in tree kernel modules but will fail > when we try to sing any out of tree kernel module. > >> > > > To sign the out of tree kernel we will use the binaries from the > shared workdir but the native libcrypto.so.3 is removed by > >> > > > the rm_work bbclass. We need to link the sign-file statically > otherwise it will not work with the rm_work bbclass. > >> > > > > >> > > > [1] > https://github.com/torvalds/linux/blob/master/scripts/sign-file.c > >> > > > > >> > > > Another solution for this problem can be changing the > make-mod-scripts to be a native tool and in this way > >> > > > they will be installed and the dependencies will be handled > correctly. > >> > > > >> > > There would very likely be different issues if the scripts were > >> > > generated and then packaged as a native tool / package. Since they > are > >> > > so tightly coupled to the kernel. We'd just trade one set of issues > >> > > for another (out of sync artifacts, etc). > > > > > > Maybe some new issues will come with this change but this will be more > aligned with > > majority of the other tools. This is something I will keep in my TODO > list. > > > >> > >> > > > >> > > I'm going to hack on this a bit. > >> > > > >> > > That being said, I've never done any module signing .. since I don't > >> > > need it in my development workflow. > >> > > > >> > > Is there a canonical guide to getting it setup so I can test my > static > >> > > link and relocated artifacts fixes ? is it with meta-integrity and > the > >> > > kernel-modsign bbclass ? > > > > > > Yes, I am using the kernel-modsign bbclass of meta-security. > > The step to needed is add modsign in DISTRO_FUTURES and > > provide the required keys setting the MODSIGN_* variable in the bbclass > > > >> > > >> > I did think about this a bit more. It does likely depend on the > version > >> > of libcrypto from the host system as to whether it reproduces or not. > >> > The possible solution ideas I came up with are: > >> > > >> > a) statically link sign-file so we don't need libcrypto > >> > b) copy the libcrypto.so into a known location in the shared workdir > >> > (probably some new path) and then adding an RPATH/RUNPATH using > chrpath > >> > to the binary. > >> > >> Agreed. I have sign-file statically building here, and it works, but > >> objtool is blowing up under static linking. > > > > > > Building the sign-file statically looks like a good solution but I have > no idea of the side effects. > > There's no side effects to the tool itself (outside of the normal "the > binary is a bit larger", etc, it is some of the other utilities that > are causing issues. > > > > >> > >> > >> If I can't break the tools into separate bits, or fix that static link > >> .. My other idea is the same as yours, if we copy out the .so and make > >> sure it is in a recognized location in the artifacts dir, we are good > >> to go. > > > > > > But I believe that for copying it will require copying the full native > sysroot dependency chain and > > not only libcrypto.so. > > The only thing missing is libcrypto (from my checking), so it doesn't > look like we'd need any more than that. But yes, it would be expected > we'd have to bring in all the requirements (but many of the common > ones are already provided by the default native environment). > > Bruce > > > > > Jose > > > >> > >> > >> Bruce > >> > >> > > >> > Cheers, > >> > > >> > Richard > >> > > >> > > >> > >> > >> -- > >> - Thou shalt not follow the NULL pointer, for chaos and madness await > >> thee at its end > >> - "Use the force Harry" - Gandalf, Star Trek II > > > > > > > > -- > > Best regards, > > > > José Quaresma > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II >
On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: > Hi, > > Not related with the previous discussion but just for > your information. > The rm_work.bbclass has an exception for the kernel recipes [1]. > So I don't understand why we can't do the same for the make-mod- > scripts > who is the twin brother of all these kernel recipes. > > [1] > https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 Ideally we wouldn't be doing this for the kernel recipes. There is also a big difference to that and the proposed patch. The proposed patch was preserving a specific directory rather than an entire recipe. Removing the task stamps but leaving a small piece of WORKDIR is quite different to preserving WORKDIR and STAMPS for a specific recipe. The former is not tested and will break things. The latter is better tolerated by bitbake. So yes, we could do the same. I'm sure there will be other recipes people want to preserve for other reasons. Where do we draw the line? We could preserve everything and drop rm_work, then we wouldn't have these problems? :) Cheers, Richard
On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: > > Hi, > > > > Not related with the previous discussion but just for > > your information. > > The rm_work.bbclass has an exception for the kernel recipes [1]. > > So I don't understand why we can't do the same for the make-mod- > > scripts > > who is the twin brother of all these kernel recipes. > > > > [1] > > https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 > > Ideally we wouldn't be doing this for the kernel recipes. > > There is also a big difference to that and the proposed patch. The > proposed patch was preserving a specific directory rather than an > entire recipe. Removing the task stamps but leaving a small piece of > WORKDIR is quite different to preserving WORKDIR and STAMPS for a > specific recipe. The former is not tested and will break things. The > latter is better tolerated by bitbake. Agreed. Plus, I am working on this now. I have static linking of the scripts/tools working, but what I haven't figured out is how to do that without patching the Makefiles. Next up will be some rpath trickery. Bruce > > So yes, we could do the same. I'm sure there will be other recipes > people want to preserve for other reasons. Where do we draw the line? > We could preserve everything and drop rm_work, then we wouldn't have > these problems? :) > > Cheers, > > Richard
On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via lists.openembedded.org <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > > On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > > > On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: > > > Hi, > > > > > > Not related with the previous discussion but just for > > > your information. > > > The rm_work.bbclass has an exception for the kernel recipes [1]. > > > So I don't understand why we can't do the same for the make-mod- > > > scripts > > > who is the twin brother of all these kernel recipes. > > > > > > [1] > > > https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 > > > > Ideally we wouldn't be doing this for the kernel recipes. > > > > There is also a big difference to that and the proposed patch. The > > proposed patch was preserving a specific directory rather than an > > entire recipe. Removing the task stamps but leaving a small piece of > > WORKDIR is quite different to preserving WORKDIR and STAMPS for a > > specific recipe. The former is not tested and will break things. The > > latter is better tolerated by bitbake. > > Agreed. > > Plus, I am working on this now. > > I have static linking of the scripts/tools working, but what I haven't > figured out is how to do that without patching the Makefiles. > It turned out to be quite the battle to get older kernels what was required for static linking of the tools. Attached is my WIP patch. I'm out of the office early next week, but will revisit it once I'm back. Bruce > Next up will be some rpath trickery. > > Bruce > > > > > So yes, we could do the same. I'm sure there will be other recipes > > people want to preserve for other reasons. Where do we draw the line? > > We could preserve everything and drop rm_work, then we wouldn't have > > these problems? :) > > > > Cheers, > > > > Richard > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180233): https://lists.openembedded.org/g/openembedded-core/message/180233 > Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Am 21.04.23 um 22:28 schrieb Bruce Ashfield: > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via > lists.openembedded.org > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: >> >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie >> <richard.purdie@linuxfoundation.org> wrote: >>> >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: >>>> Hi, >>>> >>>> Not related with the previous discussion but just for >>>> your information. >>>> The rm_work.bbclass has an exception for the kernel recipes [1]. >>>> So I don't understand why we can't do the same for the make-mod- >>>> scripts >>>> who is the twin brother of all these kernel recipes. >>>> >>>> [1] >>>> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 >>> >>> Ideally we wouldn't be doing this for the kernel recipes. >>> >>> There is also a big difference to that and the proposed patch. The >>> proposed patch was preserving a specific directory rather than an >>> entire recipe. Removing the task stamps but leaving a small piece of >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a >>> specific recipe. The former is not tested and will break things. The >>> latter is better tolerated by bitbake. >> >> Agreed. >> >> Plus, I am working on this now. >> >> I have static linking of the scripts/tools working, but what I haven't >> figured out is how to do that without patching the Makefiles. >> > > It turned out to be quite the battle to get older kernels what was > required for static linking of the tools. > > Attached is my WIP patch. I'm out of the office early next week, but > will revisit it once I'm back. > > Bruce > >> Next up will be some rpath trickery. >> >> Bruce >> >>> >>> So yes, we could do the same. I'm sure there will be other recipes >>> people want to preserve for other reasons. Where do we draw the line? >>> We could preserve everything and drop rm_work, then we wouldn't have >>> these problems? :) >>> >>> Cheers, >>> >>> Richard >> >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#180233): https://lists.openembedded.org/g/openembedded-core/message/180233 >> Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> > > Thank you for your work, I see you put some time and effort into it. HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19 (see kernel patch [1]), so we need a way to call 'pkg-config --static' with pre-5.19 kernels. A way without modifying the Makefile would be to modify openssls pkg-config in recipe-sysroot-native of make-mod-script, so 'pkg-config --libs' actually shows the dependencies of 'pkg-config --static --libs', but it's a bit hacky. Also fully-static executables still need the same glibc during runtime that they were built with, which makes them error-prone and is generally discouraged. As an alternative, we could build dynamic executables that use the static libcrypto library. The linker links by default against the shared library, so we could remove them from recipe-sysroot-native to force linking against the static library (again, somewhat hacky). [1] https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5
On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer <Christoph.Lauer@email.de> wrote: > > Am 21.04.23 um 22:28 schrieb Bruce Ashfield: > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via > > lists.openembedded.org > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie > >> <richard.purdie@linuxfoundation.org> wrote: > >>> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: > >>>> Hi, > >>>> > >>>> Not related with the previous discussion but just for > >>>> your information. > >>>> The rm_work.bbclass has an exception for the kernel recipes [1]. > >>>> So I don't understand why we can't do the same for the make-mod- > >>>> scripts > >>>> who is the twin brother of all these kernel recipes. > >>>> > >>>> [1] > >>>> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 > >>> > >>> Ideally we wouldn't be doing this for the kernel recipes. > >>> > >>> There is also a big difference to that and the proposed patch. The > >>> proposed patch was preserving a specific directory rather than an > >>> entire recipe. Removing the task stamps but leaving a small piece of > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a > >>> specific recipe. The former is not tested and will break things. The > >>> latter is better tolerated by bitbake. > >> > >> Agreed. > >> > >> Plus, I am working on this now. > >> > >> I have static linking of the scripts/tools working, but what I haven't > >> figured out is how to do that without patching the Makefiles. > >> > > > > It turned out to be quite the battle to get older kernels what was > > required for static linking of the tools. > > > > Attached is my WIP patch. I'm out of the office early next week, but > > will revisit it once I'm back. > > > > Bruce > > > >> Next up will be some rpath trickery. > >> > >> Bruce > >> > >>> > >>> So yes, we could do the same. I'm sure there will be other recipes > >>> people want to preserve for other reasons. Where do we draw the line? > >>> We could preserve everything and drop rm_work, then we wouldn't have > >>> these problems? :) > >>> > >>> Cheers, > >>> > >>> Richard > >> > >> > >> > >> -- > >> - Thou shalt not follow the NULL pointer, for chaos and madness await > >> thee at its end > >> - "Use the force Harry" - Gandalf, Star Trek II > >> > >> -=-=-=-=-=-=-=-=-=-=-=- > >> Links: You receive all messages sent to this group. > >> View/Reply Online (#180233): https://lists.openembedded.org/g/openembedded-core/message/180233 > >> Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 > >> Group Owner: openembedded-core+owner@lists.openembedded.org > >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > >> -=-=-=-=-=-=-=-=-=-=-=- > >> > > > > > > Thank you for your work, I see you put some time and effort into it. > HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19 Yes, I realize that and documented it in the patch ... but I also tested on pre-5.19 kernels and what I have in the patch works. Did it not work in your testing ? > (see kernel patch [1]), so we need a way to call 'pkg-config --static' > with pre-5.19 kernels. A way without modifying the Makefile would be to > modify openssls pkg-config in recipe-sysroot-native of make-mod-script, > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config > --static --libs', but it's a bit hacky. Already considered, and discarded. That's not going to fly. > > Also fully-static executables still need the same glibc during runtime > that they were built with, which makes them error-prone and is generally > discouraged. As an alternative, we could build dynamic executables that > use the static libcrypto library. The linker links by default against > the shared library, so we could remove them from recipe-sysroot-native > to force linking against the static library (again, somewhat hacky). Also considered and discarded. As do the dynamically linked ones for the c runtime. We aren't talking about using these outside of a single build and they are generated on the fly, so again, there's very little concern about runtimes changing after linking.. There's less risk in static than in the alternatives. Bruce > > [1] > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5
Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo, 23/04/2023 à(s) 20:55: > On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer > <Christoph.Lauer@email.de> wrote: > > > > Am 21.04.23 um 22:28 schrieb Bruce Ashfield: > > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via > > > lists.openembedded.org > > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > > >> > > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie > > >> <richard.purdie@linuxfoundation.org> wrote: > > >>> > > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: > > >>>> Hi, > > >>>> > > >>>> Not related with the previous discussion but just for > > >>>> your information. > > >>>> The rm_work.bbclass has an exception for the kernel recipes [1]. > > >>>> So I don't understand why we can't do the same for the make-mod- > > >>>> scripts > > >>>> who is the twin brother of all these kernel recipes. > > >>>> > > >>>> [1] > > >>>> > https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 > > >>> > > >>> Ideally we wouldn't be doing this for the kernel recipes. > > >>> > > >>> There is also a big difference to that and the proposed patch. The > > >>> proposed patch was preserving a specific directory rather than an > > >>> entire recipe. Removing the task stamps but leaving a small piece of > > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a > > >>> specific recipe. The former is not tested and will break things. The > > >>> latter is better tolerated by bitbake. > > >> > > >> Agreed. > > >> > > >> Plus, I am working on this now. > > >> > > >> I have static linking of the scripts/tools working, but what I haven't > > >> figured out is how to do that without patching the Makefiles. > > >> > > > > > > It turned out to be quite the battle to get older kernels what was > > > required for static linking of the tools. > > > > > > Attached is my WIP patch. I'm out of the office early next week, but > > > will revisit it once I'm back. > > > > > > Bruce > > > > > >> Next up will be some rpath trickery. > > >> > > >> Bruce > > >> > > >>> > > >>> So yes, we could do the same. I'm sure there will be other recipes > > >>> people want to preserve for other reasons. Where do we draw the line? > > >>> We could preserve everything and drop rm_work, then we wouldn't have > > >>> these problems? :) > > >>> > > >>> Cheers, > > >>> > > >>> Richard > > >> > > >> > > >> > > >> -- > > >> - Thou shalt not follow the NULL pointer, for chaos and madness await > > >> thee at its end > > >> - "Use the force Harry" - Gandalf, Star Trek II > > >> > > >> -=-=-=-=-=-=-=-=-=-=-=- > > >> Links: You receive all messages sent to this group. > > >> View/Reply Online (#180233): > https://lists.openembedded.org/g/openembedded-core/message/180233 > > >> Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 > > >> Group Owner: openembedded-core+owner@lists.openembedded.org > > >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub > [bruce.ashfield@gmail.com] > > >> -=-=-=-=-=-=-=-=-=-=-=- > > >> > > > > > > > > > > Thank you for your work, I see you put some time and effort into it. > > HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19 > > Yes, I realize that and documented it in the patch ... but I also > tested on pre-5.19 kernels and what I have in the patch works. Did it > not work in your testing ? > I will test the patch on a couple of kernel versions with some of them pre-5.19 but all in 5 major versions. I will say something about my results later this week. Thanks for working on this one. Jose > > > (see kernel patch [1]), so we need a way to call 'pkg-config --static' > > with pre-5.19 kernels. A way without modifying the Makefile would be to > > modify openssls pkg-config in recipe-sysroot-native of make-mod-script, > > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config > > --static --libs', but it's a bit hacky. > > Already considered, and discarded. That's not going to fly. > > > > > Also fully-static executables still need the same glibc during runtime > > that they were built with, which makes them error-prone and is generally > > discouraged. As an alternative, we could build dynamic executables that > > use the static libcrypto library. The linker links by default against > > the shared library, so we could remove them from recipe-sysroot-native > > to force linking against the static library (again, somewhat hacky). > > Also considered and discarded. > > As do the dynamically linked ones for the c runtime. We aren't talking > about using these outside of a single build and they are generated on > the fly, so again, there's very little concern about runtimes changing > after linking.. There's less risk in static than in the alternatives. > > Bruce > > > > > > [1] > > > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II >
On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo, 23/04/2023 à(s) 20:55: >> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer >> <Christoph.Lauer@email.de> wrote: >> > >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield: >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via >> > > lists.openembedded.org >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: >> > >> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie >> > >> <richard.purdie@linuxfoundation.org> wrote: >> > >>> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: >> > >>>> Hi, >> > >>>> >> > >>>> Not related with the previous discussion but just for >> > >>>> your information. >> > >>>> The rm_work.bbclass has an exception for the kernel recipes [1]. >> > >>>> So I don't understand why we can't do the same for the make-mod- >> > >>>> scripts >> > >>>> who is the twin brother of all these kernel recipes. >> > >>>> >> > >>>> [1] >> > >>>> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 >> > >>> >> > >>> Ideally we wouldn't be doing this for the kernel recipes. >> > >>> >> > >>> There is also a big difference to that and the proposed patch. The >> > >>> proposed patch was preserving a specific directory rather than an >> > >>> entire recipe. Removing the task stamps but leaving a small piece of >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a >> > >>> specific recipe. The former is not tested and will break things. The >> > >>> latter is better tolerated by bitbake. >> > >> >> > >> Agreed. >> > >> >> > >> Plus, I am working on this now. >> > >> >> > >> I have static linking of the scripts/tools working, but what I haven't >> > >> figured out is how to do that without patching the Makefiles. >> > >> >> > > >> > > It turned out to be quite the battle to get older kernels what was >> > > required for static linking of the tools. >> > > >> > > Attached is my WIP patch. I'm out of the office early next week, but >> > > will revisit it once I'm back. >> > > >> > > Bruce >> > > >> > >> Next up will be some rpath trickery. >> > >> >> > >> Bruce >> > >> >> > >>> >> > >>> So yes, we could do the same. I'm sure there will be other recipes >> > >>> people want to preserve for other reasons. Where do we draw the line? >> > >>> We could preserve everything and drop rm_work, then we wouldn't have >> > >>> these problems? :) >> > >>> >> > >>> Cheers, >> > >>> >> > >>> Richard >> > >> >> > >> >> > >> >> > >> -- >> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> > >> thee at its end >> > >> - "Use the force Harry" - Gandalf, Star Trek II >> > >> >> > >> -=-=-=-=-=-=-=-=-=-=-=- >> > >> Links: You receive all messages sent to this group. >> > >> View/Reply Online (#180233): https://lists.openembedded.org/g/openembedded-core/message/180233 >> > >> Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 >> > >> Group Owner: openembedded-core+owner@lists.openembedded.org >> > >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] >> > >> -=-=-=-=-=-=-=-=-=-=-=- >> > >> >> > > >> > > >> > >> > Thank you for your work, I see you put some time and effort into it. >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19 >> >> Yes, I realize that and documented it in the patch ... but I also >> tested on pre-5.19 kernels and what I have in the patch works. Did it >> not work in your testing ? > > > I will test the patch on a couple of kernel versions with some of them pre-5.19 > but all in 5 major versions. > I will say something about my results later this week. 5.15-stable also has the pkg-config changes Bruce > > Thanks for working on this one. > > Jose > >> >> >> > (see kernel patch [1]), so we need a way to call 'pkg-config --static' >> > with pre-5.19 kernels. A way without modifying the Makefile would be to >> > modify openssls pkg-config in recipe-sysroot-native of make-mod-script, >> > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config >> > --static --libs', but it's a bit hacky. >> >> Already considered, and discarded. That's not going to fly. >> >> > >> > Also fully-static executables still need the same glibc during runtime >> > that they were built with, which makes them error-prone and is generally >> > discouraged. As an alternative, we could build dynamic executables that >> > use the static libcrypto library. The linker links by default against >> > the shared library, so we could remove them from recipe-sysroot-native >> > to force linking against the static library (again, somewhat hacky). >> >> Also considered and discarded. >> >> As do the dynamically linked ones for the c runtime. We aren't talking >> about using these outside of a single build and they are generated on >> the fly, so again, there's very little concern about runtimes changing >> after linking.. There's less risk in static than in the alternatives. >> >> Bruce >> >> >> > >> > [1] >> > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 >> >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II > > > > -- > Best regards, > > José Quaresma
Hi Bruce,
I have been testing your patch and have some comments.
In some of my kernels I don't have the pkg-config changes and so I have
some fails linking the scripts/sign-file
because for static linking with the libcrypto we need the -ldl -pthread.
To fix my build I need to override the CRYPTO_LIBS in my kernel because
they use the hardcoded pkg-config
where it is not possible to pass the --static argument.
With following change on top of your patch I can build moist of my kernels:
export HOSTLDFLAGS="-lz"
+ HOSTPKG_CONFIG="pkg-config --static"
+ # override CRYPTO_LIBS since HOSTPKG_CONFIG lands only in v5.19-rc1
+ #
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5ea4fece4508bf8e72b659cd22fa4840d8d61e5
+ CRYPTO_LIBS="$(pkg-config --static --libs libcrypto 2>/dev/null ||
echo -lcrypto)"
+
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
for t in prepare scripts_basic scripts; do
oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \
- HOSTPKG_CONFIG="pkg-config --static" \
+ HOSTPKG_CONFIG="${HOSTPKG_CONFIG}"
CRYPTO_LIBS="${CRYPTO_LIBS}" \
-C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
done
I think belive that the LIBELF_LIBS needs the same fix for the cases where
HOSTPKG_CONFIG is not available.
Also I think there is a typo in the LIBELF_LIBS because you first populate
it with pkg-config but on the export the variable is redefined
with the HOST_LIBELF_LIBS. I made a litle change too on that:
# for pre-5.15 kernels
- LIBELF_LIBS=$(pkg-config libelf --libs 2>/dev/null || echo -lelf)
- export LIBELF_LIBS="$HOST_LIBELF_LIBS -lz"
+ LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null || echo
-lelf)
+ export LIBELF_LIBS="$LIBELF_LIBS -lz"
export HOSTLDFLAGS="-lz"
Thanks for you help.
Jose
Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia segunda,
24/04/2023 à(s) 20:25:
> On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma <quaresma.jose@gmail.com>
> wrote:
> >
> >
> >
> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo,
> 23/04/2023 à(s) 20:55:
> >>
> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer
> >> <Christoph.Lauer@email.de> wrote:
> >> >
> >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield:
> >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via
> >> > > lists.openembedded.org
> >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
> >> > >>
> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie
> >> > >> <richard.purdie@linuxfoundation.org> wrote:
> >> > >>>
> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote:
> >> > >>>> Hi,
> >> > >>>>
> >> > >>>> Not related with the previous discussion but just for
> >> > >>>> your information.
> >> > >>>> The rm_work.bbclass has an exception for the kernel recipes [1].
> >> > >>>> So I don't understand why we can't do the same for the make-mod-
> >> > >>>> scripts
> >> > >>>> who is the twin brother of all these kernel recipes.
> >> > >>>>
> >> > >>>> [1]
> >> > >>>>
> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168
> >> > >>>
> >> > >>> Ideally we wouldn't be doing this for the kernel recipes.
> >> > >>>
> >> > >>> There is also a big difference to that and the proposed patch. The
> >> > >>> proposed patch was preserving a specific directory rather than an
> >> > >>> entire recipe. Removing the task stamps but leaving a small piece
> of
> >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a
> >> > >>> specific recipe. The former is not tested and will break things.
> The
> >> > >>> latter is better tolerated by bitbake.
> >> > >>
> >> > >> Agreed.
> >> > >>
> >> > >> Plus, I am working on this now.
> >> > >>
> >> > >> I have static linking of the scripts/tools working, but what I
> haven't
> >> > >> figured out is how to do that without patching the Makefiles.
> >> > >>
> >> > >
> >> > > It turned out to be quite the battle to get older kernels what was
> >> > > required for static linking of the tools.
> >> > >
> >> > > Attached is my WIP patch. I'm out of the office early next week, but
> >> > > will revisit it once I'm back.
> >> > >
> >> > > Bruce
> >> > >
> >> > >> Next up will be some rpath trickery.
> >> > >>
> >> > >> Bruce
> >> > >>
> >> > >>>
> >> > >>> So yes, we could do the same. I'm sure there will be other recipes
> >> > >>> people want to preserve for other reasons. Where do we draw the
> line?
> >> > >>> We could preserve everything and drop rm_work, then we wouldn't
> have
> >> > >>> these problems? :)
> >> > >>>
> >> > >>> Cheers,
> >> > >>>
> >> > >>> Richard
> >> > >>
> >> > >>
> >> > >>
> >> > >> --
> >> > >> - Thou shalt not follow the NULL pointer, for chaos and madness
> await
> >> > >> thee at its end
> >> > >> - "Use the force Harry" - Gandalf, Star Trek II
> >> > >>
> >> > >> -=-=-=-=-=-=-=-=-=-=-=-
> >> > >> Links: You receive all messages sent to this group.
> >> > >> View/Reply Online (#180233):
> https://lists.openembedded.org/g/openembedded-core/message/180233
> >> > >> Mute This Topic:
> https://lists.openembedded.org/mt/98296212/1050810
> >> > >> Group Owner: openembedded-core+owner@lists.openembedded.org
> >> > >> Unsubscribe:
> https://lists.openembedded.org/g/openembedded-core/unsub [
> bruce.ashfield@gmail.com]
> >> > >> -=-=-=-=-=-=-=-=-=-=-=-
> >> > >>
> >> > >
> >> > >
> >> >
> >> > Thank you for your work, I see you put some time and effort into it.
> >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel version
> 5.19
> >>
> >> Yes, I realize that and documented it in the patch ... but I also
> >> tested on pre-5.19 kernels and what I have in the patch works. Did it
> >> not work in your testing ?
> >
> >
> > I will test the patch on a couple of kernel versions with some of them
> pre-5.19
> > but all in 5 major versions.
> > I will say something about my results later this week.
>
> 5.15-stable also has the pkg-config changes
>
> Bruce
>
> >
> > Thanks for working on this one.
> >
> > Jose
> >
> >>
> >>
> >> > (see kernel patch [1]), so we need a way to call 'pkg-config --static'
> >> > with pre-5.19 kernels. A way without modifying the Makefile would be
> to
> >> > modify openssls pkg-config in recipe-sysroot-native of
> make-mod-script,
> >> > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config
> >> > --static --libs', but it's a bit hacky.
> >>
> >> Already considered, and discarded. That's not going to fly.
> >>
> >> >
> >> > Also fully-static executables still need the same glibc during runtime
> >> > that they were built with, which makes them error-prone and is
> generally
> >> > discouraged. As an alternative, we could build dynamic executables
> that
> >> > use the static libcrypto library. The linker links by default against
> >> > the shared library, so we could remove them from recipe-sysroot-native
> >> > to force linking against the static library (again, somewhat hacky).
> >>
> >> Also considered and discarded.
> >>
> >> As do the dynamically linked ones for the c runtime. We aren't talking
> >> about using these outside of a single build and they are generated on
> >> the fly, so again, there's very little concern about runtimes changing
> >> after linking.. There's less risk in static than in the alternatives.
> >>
> >> Bruce
> >>
> >>
> >> >
> >> > [1]
> >> >
> https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5
> >>
> >>
> >>
> >> --
> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> thee at its end
> >> - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > --
> > Best regards,
> >
> > José Quaresma
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
On Thu, Apr 27, 2023 at 6:32 PM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > Hi Bruce, > > I have been testing your patch and have some comments. > In some of my kernels I don't have the pkg-config changes and so I have some fails linking the scripts/sign-file > because for static linking with the libcrypto we need the -ldl -pthread. > > To fix my build I need to override the CRYPTO_LIBS in my kernel because they use the hardcoded pkg-config > where it is not possible to pass the --static argument. > > With following change on top of your patch I can build moist of my kernels: > > export HOSTLDFLAGS="-lz" > > + HOSTPKG_CONFIG="pkg-config --static" > + # override CRYPTO_LIBS since HOSTPKG_CONFIG lands only in v5.19-rc1 > + # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 > + CRYPTO_LIBS="$(pkg-config --static --libs libcrypto 2>/dev/null || echo -lcrypto)" > + > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS > for t in prepare scripts_basic scripts; do > oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \ > AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \ > - HOSTPKG_CONFIG="pkg-config --static" \ > + HOSTPKG_CONFIG="${HOSTPKG_CONFIG}" CRYPTO_LIBS="${CRYPTO_LIBS}" \ > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > done > > > I think belive that the LIBELF_LIBS needs the same fix for the cases where HOSTPKG_CONFIG is not available. > Also I think there is a typo in the LIBELF_LIBS because you first populate it with pkg-config but on the export the variable is redefined > with the HOST_LIBELF_LIBS. I made a litle change too on that: > > # for pre-5.15 kernels > - LIBELF_LIBS=$(pkg-config libelf --libs 2>/dev/null || echo -lelf) > - export LIBELF_LIBS="$HOST_LIBELF_LIBS -lz" > + LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null || echo -lelf) > + export LIBELF_LIBS="$LIBELF_LIBS -lz" > export HOSTLDFLAGS="-lz" > > Thanks for you help. Those are definitely plausible tweaks to the patch, I was providing the two techniques for that reason, and you've used them appropriately. Let me roll your changes into my patch, re-test and I'll submit it to the mailing list as a v2. Thanks for the testing, and fixup, I knew there would be things missing! :) Bruce > > Jose > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia segunda, 24/04/2023 à(s) 20:25: >> >> On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma <quaresma.jose@gmail.com> wrote: >> > >> > >> > >> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo, 23/04/2023 à(s) 20:55: >> >> >> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer >> >> <Christoph.Lauer@email.de> wrote: >> >> > >> >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield: >> >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via >> >> > > lists.openembedded.org >> >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: >> >> > >> >> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie >> >> > >> <richard.purdie@linuxfoundation.org> wrote: >> >> > >>> >> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: >> >> > >>>> Hi, >> >> > >>>> >> >> > >>>> Not related with the previous discussion but just for >> >> > >>>> your information. >> >> > >>>> The rm_work.bbclass has an exception for the kernel recipes [1]. >> >> > >>>> So I don't understand why we can't do the same for the make-mod- >> >> > >>>> scripts >> >> > >>>> who is the twin brother of all these kernel recipes. >> >> > >>>> >> >> > >>>> [1] >> >> > >>>> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 >> >> > >>> >> >> > >>> Ideally we wouldn't be doing this for the kernel recipes. >> >> > >>> >> >> > >>> There is also a big difference to that and the proposed patch. The >> >> > >>> proposed patch was preserving a specific directory rather than an >> >> > >>> entire recipe. Removing the task stamps but leaving a small piece of >> >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a >> >> > >>> specific recipe. The former is not tested and will break things. The >> >> > >>> latter is better tolerated by bitbake. >> >> > >> >> >> > >> Agreed. >> >> > >> >> >> > >> Plus, I am working on this now. >> >> > >> >> >> > >> I have static linking of the scripts/tools working, but what I haven't >> >> > >> figured out is how to do that without patching the Makefiles. >> >> > >> >> >> > > >> >> > > It turned out to be quite the battle to get older kernels what was >> >> > > required for static linking of the tools. >> >> > > >> >> > > Attached is my WIP patch. I'm out of the office early next week, but >> >> > > will revisit it once I'm back. >> >> > > >> >> > > Bruce >> >> > > >> >> > >> Next up will be some rpath trickery. >> >> > >> >> >> > >> Bruce >> >> > >> >> >> > >>> >> >> > >>> So yes, we could do the same. I'm sure there will be other recipes >> >> > >>> people want to preserve for other reasons. Where do we draw the line? >> >> > >>> We could preserve everything and drop rm_work, then we wouldn't have >> >> > >>> these problems? :) >> >> > >>> >> >> > >>> Cheers, >> >> > >>> >> >> > >>> Richard >> >> > >> >> >> > >> >> >> > >> >> >> > >> -- >> >> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> >> > >> thee at its end >> >> > >> - "Use the force Harry" - Gandalf, Star Trek II >> >> > >> >> >> > >> -=-=-=-=-=-=-=-=-=-=-=- >> >> > >> Links: You receive all messages sent to this group. >> >> > >> View/Reply Online (#180233): https://lists.openembedded.org/g/openembedded-core/message/180233 >> >> > >> Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 >> >> > >> Group Owner: openembedded-core+owner@lists.openembedded.org >> >> > >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] >> >> > >> -=-=-=-=-=-=-=-=-=-=-=- >> >> > >> >> >> > > >> >> > > >> >> > >> >> > Thank you for your work, I see you put some time and effort into it. >> >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19 >> >> >> >> Yes, I realize that and documented it in the patch ... but I also >> >> tested on pre-5.19 kernels and what I have in the patch works. Did it >> >> not work in your testing ? >> > >> > >> > I will test the patch on a couple of kernel versions with some of them pre-5.19 >> > but all in 5 major versions. >> > I will say something about my results later this week. >> >> 5.15-stable also has the pkg-config changes >> >> Bruce >> >> > >> > Thanks for working on this one. >> > >> > Jose >> > >> >> >> >> >> >> > (see kernel patch [1]), so we need a way to call 'pkg-config --static' >> >> > with pre-5.19 kernels. A way without modifying the Makefile would be to >> >> > modify openssls pkg-config in recipe-sysroot-native of make-mod-script, >> >> > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config >> >> > --static --libs', but it's a bit hacky. >> >> >> >> Already considered, and discarded. That's not going to fly. >> >> >> >> > >> >> > Also fully-static executables still need the same glibc during runtime >> >> > that they were built with, which makes them error-prone and is generally >> >> > discouraged. As an alternative, we could build dynamic executables that >> >> > use the static libcrypto library. The linker links by default against >> >> > the shared library, so we could remove them from recipe-sysroot-native >> >> > to force linking against the static library (again, somewhat hacky). >> >> >> >> Also considered and discarded. >> >> >> >> As do the dynamically linked ones for the c runtime. We aren't talking >> >> about using these outside of a single build and they are generated on >> >> the fly, so again, there's very little concern about runtimes changing >> >> after linking.. There's less risk in static than in the alternatives. >> >> >> >> Bruce >> >> >> >> >> >> > >> >> > [1] >> >> > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 >> >> >> >> >> >> >> >> -- >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> >> thee at its end >> >> - "Use the force Harry" - Gandalf, Star Trek II >> > >> > >> > >> > -- >> > Best regards, >> > >> > José Quaresma >> >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II > > > > -- > Best regards, > > José Quaresma
Attached is v2 of the patch. I've consolidated the suggested changes. I'm soaking it a bit longer, and then will send it as part of my next consolidated pull request. Bruce On Thu, Apr 27, 2023 at 9:26 PM Bruce Ashfield via lists.openembedded.org <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > > On Thu, Apr 27, 2023 at 6:32 PM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > > > Hi Bruce, > > > > I have been testing your patch and have some comments. > > In some of my kernels I don't have the pkg-config changes and so I have some fails linking the scripts/sign-file > > because for static linking with the libcrypto we need the -ldl -pthread. > > > > To fix my build I need to override the CRYPTO_LIBS in my kernel because they use the hardcoded pkg-config > > where it is not possible to pass the --static argument. > > > > With following change on top of your patch I can build moist of my kernels: > > > > export HOSTLDFLAGS="-lz" > > > > + HOSTPKG_CONFIG="pkg-config --static" > > + # override CRYPTO_LIBS since HOSTPKG_CONFIG lands only in v5.19-rc1 > > + # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 > > + CRYPTO_LIBS="$(pkg-config --static --libs libcrypto 2>/dev/null || echo -lcrypto)" > > + > > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS > > for t in prepare scripts_basic scripts; do > > oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \ > > AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \ > > - HOSTPKG_CONFIG="pkg-config --static" \ > > + HOSTPKG_CONFIG="${HOSTPKG_CONFIG}" CRYPTO_LIBS="${CRYPTO_LIBS}" \ > > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t > > done > > > > > > I think belive that the LIBELF_LIBS needs the same fix for the cases where HOSTPKG_CONFIG is not available. > > Also I think there is a typo in the LIBELF_LIBS because you first populate it with pkg-config but on the export the variable is redefined > > with the HOST_LIBELF_LIBS. I made a litle change too on that: > > > > # for pre-5.15 kernels > > - LIBELF_LIBS=$(pkg-config libelf --libs 2>/dev/null || echo -lelf) > > - export LIBELF_LIBS="$HOST_LIBELF_LIBS -lz" > > + LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null || echo -lelf) > > + export LIBELF_LIBS="$LIBELF_LIBS -lz" > > export HOSTLDFLAGS="-lz" > > > > Thanks for you help. > > Those are definitely plausible tweaks to the patch, I was providing > the two techniques for that reason, and you've used them > appropriately. > > Let me roll your changes into my patch, re-test and I'll submit it to > the mailing list as a v2. > > Thanks for the testing, and fixup, I knew there would be things missing! :) > > Bruce > > > > > Jose > > > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia segunda, 24/04/2023 à(s) 20:25: > >> > >> On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma <quaresma.jose@gmail.com> wrote: > >> > > >> > > >> > > >> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo, 23/04/2023 à(s) 20:55: > >> >> > >> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer > >> >> <Christoph.Lauer@email.de> wrote: > >> >> > > >> >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield: > >> >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via > >> >> > > lists.openembedded.org > >> >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > >> >> > >> > >> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie > >> >> > >> <richard.purdie@linuxfoundation.org> wrote: > >> >> > >>> > >> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: > >> >> > >>>> Hi, > >> >> > >>>> > >> >> > >>>> Not related with the previous discussion but just for > >> >> > >>>> your information. > >> >> > >>>> The rm_work.bbclass has an exception for the kernel recipes [1]. > >> >> > >>>> So I don't understand why we can't do the same for the make-mod- > >> >> > >>>> scripts > >> >> > >>>> who is the twin brother of all these kernel recipes. > >> >> > >>>> > >> >> > >>>> [1] > >> >> > >>>> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 > >> >> > >>> > >> >> > >>> Ideally we wouldn't be doing this for the kernel recipes. > >> >> > >>> > >> >> > >>> There is also a big difference to that and the proposed patch. The > >> >> > >>> proposed patch was preserving a specific directory rather than an > >> >> > >>> entire recipe. Removing the task stamps but leaving a small piece of > >> >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a > >> >> > >>> specific recipe. The former is not tested and will break things. The > >> >> > >>> latter is better tolerated by bitbake. > >> >> > >> > >> >> > >> Agreed. > >> >> > >> > >> >> > >> Plus, I am working on this now. > >> >> > >> > >> >> > >> I have static linking of the scripts/tools working, but what I haven't > >> >> > >> figured out is how to do that without patching the Makefiles. > >> >> > >> > >> >> > > > >> >> > > It turned out to be quite the battle to get older kernels what was > >> >> > > required for static linking of the tools. > >> >> > > > >> >> > > Attached is my WIP patch. I'm out of the office early next week, but > >> >> > > will revisit it once I'm back. > >> >> > > > >> >> > > Bruce > >> >> > > > >> >> > >> Next up will be some rpath trickery. > >> >> > >> > >> >> > >> Bruce > >> >> > >> > >> >> > >>> > >> >> > >>> So yes, we could do the same. I'm sure there will be other recipes > >> >> > >>> people want to preserve for other reasons. Where do we draw the line? > >> >> > >>> We could preserve everything and drop rm_work, then we wouldn't have > >> >> > >>> these problems? :) > >> >> > >>> > >> >> > >>> Cheers, > >> >> > >>> > >> >> > >>> Richard > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> -- > >> >> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await > >> >> > >> thee at its end > >> >> > >> - "Use the force Harry" - Gandalf, Star Trek II > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > > > >> >> > > > >> >> > > >> >> > Thank you for your work, I see you put some time and effort into it. > >> >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19 > >> >> > >> >> Yes, I realize that and documented it in the patch ... but I also > >> >> tested on pre-5.19 kernels and what I have in the patch works. Did it > >> >> not work in your testing ? > >> > > >> > > >> > I will test the patch on a couple of kernel versions with some of them pre-5.19 > >> > but all in 5 major versions. > >> > I will say something about my results later this week. > >> > >> 5.15-stable also has the pkg-config changes > >> > >> Bruce > >> > >> > > >> > Thanks for working on this one. > >> > > >> > Jose > >> > > >> >> > >> >> > >> >> > (see kernel patch [1]), so we need a way to call 'pkg-config --static' > >> >> > with pre-5.19 kernels. A way without modifying the Makefile would be to > >> >> > modify openssls pkg-config in recipe-sysroot-native of make-mod-script, > >> >> > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config > >> >> > --static --libs', but it's a bit hacky. > >> >> > >> >> Already considered, and discarded. That's not going to fly. > >> >> > >> >> > > >> >> > Also fully-static executables still need the same glibc during runtime > >> >> > that they were built with, which makes them error-prone and is generally > >> >> > discouraged. As an alternative, we could build dynamic executables that > >> >> > use the static libcrypto library. The linker links by default against > >> >> > the shared library, so we could remove them from recipe-sysroot-native > >> >> > to force linking against the static library (again, somewhat hacky). > >> >> > >> >> Also considered and discarded. > >> >> > >> >> As do the dynamically linked ones for the c runtime. We aren't talking > >> >> about using these outside of a single build and they are generated on > >> >> the fly, so again, there's very little concern about runtimes changing > >> >> after linking.. There's less risk in static than in the alternatives. > >> >> > >> >> Bruce > >> >> > >> >> > >> >> > > >> >> > [1] > >> >> > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 > >> >> > >> >> > >> >> > >> >> -- > >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await > >> >> thee at its end > >> >> - "Use the force Harry" - Gandalf, Star Trek II > >> > > >> > > >> > > >> > -- > >> > Best regards, > >> > > >> > José Quaresma > >> > >> > >> > >> -- > >> - Thou shalt not follow the NULL pointer, for chaos and madness await > >> thee at its end > >> - "Use the force Harry" - Gandalf, Star Trek II > > > > > > > > -- > > Best regards, > > > > José Quaresma > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180502): https://lists.openembedded.org/g/openembedded-core/message/180502 > Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, 2/05/2023 à(s) 22:12: > Attached is v2 of the patch. I've consolidated the suggested changes. > > I'm soaking it a bit longer, and then will send it as part of my next > consolidated pull request. > I will do some more tests with the v2 and post my comment later if anything new comes up. Jose > > Bruce > > On Thu, Apr 27, 2023 at 9:26 PM Bruce Ashfield via > lists.openembedded.org > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > > > > On Thu, Apr 27, 2023 at 6:32 PM Jose Quaresma <quaresma.jose@gmail.com> > wrote: > > > > > > Hi Bruce, > > > > > > I have been testing your patch and have some comments. > > > In some of my kernels I don't have the pkg-config changes and so I > have some fails linking the scripts/sign-file > > > because for static linking with the libcrypto we need the -ldl > -pthread. > > > > > > To fix my build I need to override the CRYPTO_LIBS in my kernel > because they use the hardcoded pkg-config > > > where it is not possible to pass the --static argument. > > > > > > With following change on top of your patch I can build moist of my > kernels: > > > > > > export HOSTLDFLAGS="-lz" > > > > > > + HOSTPKG_CONFIG="pkg-config --static" > > > + # override CRYPTO_LIBS since HOSTPKG_CONFIG lands only in > v5.19-rc1 > > > + # > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 > > > + CRYPTO_LIBS="$(pkg-config --static --libs libcrypto > 2>/dev/null || echo -lcrypto)" > > > + > > > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS > > > for t in prepare scripts_basic scripts; do > > > oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \ > > > AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \ > > > - HOSTPKG_CONFIG="pkg-config --static" \ > > > + HOSTPKG_CONFIG="${HOSTPKG_CONFIG}" > CRYPTO_LIBS="${CRYPTO_LIBS}" \ > > > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} > $t > > > done > > > > > > > > > I think belive that the LIBELF_LIBS needs the same fix for the cases > where HOSTPKG_CONFIG is not available. > > > Also I think there is a typo in the LIBELF_LIBS because you first > populate it with pkg-config but on the export the variable is redefined > > > with the HOST_LIBELF_LIBS. I made a litle change too on that: > > > > > > # for pre-5.15 kernels > > > - LIBELF_LIBS=$(pkg-config libelf --libs 2>/dev/null || echo > -lelf) > > > - export LIBELF_LIBS="$HOST_LIBELF_LIBS -lz" > > > + LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null || > echo -lelf) > > > + export LIBELF_LIBS="$LIBELF_LIBS -lz" > > > export HOSTLDFLAGS="-lz" > > > > > > Thanks for you help. > > > > Those are definitely plausible tweaks to the patch, I was providing > > the two techniques for that reason, and you've used them > > appropriately. > > > > Let me roll your changes into my patch, re-test and I'll submit it to > > the mailing list as a v2. > > > > Thanks for the testing, and fixup, I knew there would be things missing! > :) > > > > Bruce > > > > > > > > Jose > > > > > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia segunda, > 24/04/2023 à(s) 20:25: > > >> > > >> On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma < > quaresma.jose@gmail.com> wrote: > > >> > > > >> > > > >> > > > >> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo, > 23/04/2023 à(s) 20:55: > > >> >> > > >> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer > > >> >> <Christoph.Lauer@email.de> wrote: > > >> >> > > > >> >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield: > > >> >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via > > >> >> > > lists.openembedded.org > > >> >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > > >> >> > >> > > >> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie > > >> >> > >> <richard.purdie@linuxfoundation.org> wrote: > > >> >> > >>> > > >> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: > > >> >> > >>>> Hi, > > >> >> > >>>> > > >> >> > >>>> Not related with the previous discussion but just for > > >> >> > >>>> your information. > > >> >> > >>>> The rm_work.bbclass has an exception for the kernel recipes > [1]. > > >> >> > >>>> So I don't understand why we can't do the same for the > make-mod- > > >> >> > >>>> scripts > > >> >> > >>>> who is the twin brother of all these kernel recipes. > > >> >> > >>>> > > >> >> > >>>> [1] > > >> >> > >>>> > https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 > > >> >> > >>> > > >> >> > >>> Ideally we wouldn't be doing this for the kernel recipes. > > >> >> > >>> > > >> >> > >>> There is also a big difference to that and the proposed > patch. The > > >> >> > >>> proposed patch was preserving a specific directory rather > than an > > >> >> > >>> entire recipe. Removing the task stamps but leaving a small > piece of > > >> >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS > for a > > >> >> > >>> specific recipe. The former is not tested and will break > things. The > > >> >> > >>> latter is better tolerated by bitbake. > > >> >> > >> > > >> >> > >> Agreed. > > >> >> > >> > > >> >> > >> Plus, I am working on this now. > > >> >> > >> > > >> >> > >> I have static linking of the scripts/tools working, but what > I haven't > > >> >> > >> figured out is how to do that without patching the Makefiles. > > >> >> > >> > > >> >> > > > > >> >> > > It turned out to be quite the battle to get older kernels what > was > > >> >> > > required for static linking of the tools. > > >> >> > > > > >> >> > > Attached is my WIP patch. I'm out of the office early next > week, but > > >> >> > > will revisit it once I'm back. > > >> >> > > > > >> >> > > Bruce > > >> >> > > > > >> >> > >> Next up will be some rpath trickery. > > >> >> > >> > > >> >> > >> Bruce > > >> >> > >> > > >> >> > >>> > > >> >> > >>> So yes, we could do the same. I'm sure there will be other > recipes > > >> >> > >>> people want to preserve for other reasons. Where do we draw > the line? > > >> >> > >>> We could preserve everything and drop rm_work, then we > wouldn't have > > >> >> > >>> these problems? :) > > >> >> > >>> > > >> >> > >>> Cheers, > > >> >> > >>> > > >> >> > >>> Richard > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> -- > > >> >> > >> - Thou shalt not follow the NULL pointer, for chaos and > madness await > > >> >> > >> thee at its end > > >> >> > >> - "Use the force Harry" - Gandalf, Star Trek II > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > > > > >> >> > > > > >> >> > > > >> >> > Thank you for your work, I see you put some time and effort into > it. > > >> >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel > version 5.19 > > >> >> > > >> >> Yes, I realize that and documented it in the patch ... but I also > > >> >> tested on pre-5.19 kernels and what I have in the patch works. Did > it > > >> >> not work in your testing ? > > >> > > > >> > > > >> > I will test the patch on a couple of kernel versions with some of > them pre-5.19 > > >> > but all in 5 major versions. > > >> > I will say something about my results later this week. > > >> > > >> 5.15-stable also has the pkg-config changes > > >> > > >> Bruce > > >> > > >> > > > >> > Thanks for working on this one. > > >> > > > >> > Jose > > >> > > > >> >> > > >> >> > > >> >> > (see kernel patch [1]), so we need a way to call 'pkg-config > --static' > > >> >> > with pre-5.19 kernels. A way without modifying the Makefile > would be to > > >> >> > modify openssls pkg-config in recipe-sysroot-native of > make-mod-script, > > >> >> > so 'pkg-config --libs' actually shows the dependencies of > 'pkg-config > > >> >> > --static --libs', but it's a bit hacky. > > >> >> > > >> >> Already considered, and discarded. That's not going to fly. > > >> >> > > >> >> > > > >> >> > Also fully-static executables still need the same glibc during > runtime > > >> >> > that they were built with, which makes them error-prone and is > generally > > >> >> > discouraged. As an alternative, we could build dynamic > executables that > > >> >> > use the static libcrypto library. The linker links by default > against > > >> >> > the shared library, so we could remove them from > recipe-sysroot-native > > >> >> > to force linking against the static library (again, somewhat > hacky). > > >> >> > > >> >> Also considered and discarded. > > >> >> > > >> >> As do the dynamically linked ones for the c runtime. We aren't > talking > > >> >> about using these outside of a single build and they are generated > on > > >> >> the fly, so again, there's very little concern about runtimes > changing > > >> >> after linking.. There's less risk in static than in the > alternatives. > > >> >> > > >> >> Bruce > > >> >> > > >> >> > > >> >> > > > >> >> > [1] > > >> >> > > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 > > >> >> > > >> >> > > >> >> > > >> >> -- > > >> >> - Thou shalt not follow the NULL pointer, for chaos and madness > await > > >> >> thee at its end > > >> >> - "Use the force Harry" - Gandalf, Star Trek II > > >> > > > >> > > > >> > > > >> > -- > > >> > Best regards, > > >> > > > >> > José Quaresma > > >> > > >> > > >> > > >> -- > > >> - Thou shalt not follow the NULL pointer, for chaos and madness await > > >> thee at its end > > >> - "Use the force Harry" - Gandalf, Star Trek II > > > > > > > > > > > > -- > > > Best regards, > > > > > > José Quaresma > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#180502): > https://lists.openembedded.org/g/openembedded-core/message/180502 > > Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > bruce.ashfield@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II >
Hi Bruce, Jose Quaresma via lists.openembedded.org <quaresma.jose= gmail.com@lists.openembedded.org> escreveu no dia quarta, 3/05/2023 à(s) 11:09: > > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, > 2/05/2023 à(s) 22:12: > >> Attached is v2 of the patch. I've consolidated the suggested changes. >> >> I'm soaking it a bit longer, and then will send it as part of my next >> consolidated pull request. >> > > I will do some more tests with the v2 and post my comment later if > anything new comes up. > Nothing new and the v2 patch works pretty well in my kernels. Jose > > Jose > > >> >> Bruce >> >> On Thu, Apr 27, 2023 at 9:26 PM Bruce Ashfield via >> lists.openembedded.org >> <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: >> > >> > On Thu, Apr 27, 2023 at 6:32 PM Jose Quaresma <quaresma.jose@gmail.com> >> wrote: >> > > >> > > Hi Bruce, >> > > >> > > I have been testing your patch and have some comments. >> > > In some of my kernels I don't have the pkg-config changes and so I >> have some fails linking the scripts/sign-file >> > > because for static linking with the libcrypto we need the -ldl >> -pthread. >> > > >> > > To fix my build I need to override the CRYPTO_LIBS in my kernel >> because they use the hardcoded pkg-config >> > > where it is not possible to pass the --static argument. >> > > >> > > With following change on top of your patch I can build moist of my >> kernels: >> > > >> > > export HOSTLDFLAGS="-lz" >> > > >> > > + HOSTPKG_CONFIG="pkg-config --static" >> > > + # override CRYPTO_LIBS since HOSTPKG_CONFIG lands only in >> v5.19-rc1 >> > > + # >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 >> > > + CRYPTO_LIBS="$(pkg-config --static --libs libcrypto >> 2>/dev/null || echo -lcrypto)" >> > > + >> > > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS >> > > for t in prepare scripts_basic scripts; do >> > > oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \ >> > > AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \ >> > > - HOSTPKG_CONFIG="pkg-config --static" \ >> > > + HOSTPKG_CONFIG="${HOSTPKG_CONFIG}" >> CRYPTO_LIBS="${CRYPTO_LIBS}" \ >> > > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} >> $t >> > > done >> > > >> > > >> > > I think belive that the LIBELF_LIBS needs the same fix for the cases >> where HOSTPKG_CONFIG is not available. >> > > Also I think there is a typo in the LIBELF_LIBS because you first >> populate it with pkg-config but on the export the variable is redefined >> > > with the HOST_LIBELF_LIBS. I made a litle change too on that: >> > > >> > > # for pre-5.15 kernels >> > > - LIBELF_LIBS=$(pkg-config libelf --libs 2>/dev/null || echo >> -lelf) >> > > - export LIBELF_LIBS="$HOST_LIBELF_LIBS -lz" >> > > + LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null >> || echo -lelf) >> > > + export LIBELF_LIBS="$LIBELF_LIBS -lz" >> > > export HOSTLDFLAGS="-lz" >> > > >> > > Thanks for you help. >> > >> > Those are definitely plausible tweaks to the patch, I was providing >> > the two techniques for that reason, and you've used them >> > appropriately. >> > >> > Let me roll your changes into my patch, re-test and I'll submit it to >> > the mailing list as a v2. >> > >> > Thanks for the testing, and fixup, I knew there would be things >> missing! :) >> > >> > Bruce >> > >> > > >> > > Jose >> > > >> > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia segunda, >> 24/04/2023 à(s) 20:25: >> > >> >> > >> On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma < >> quaresma.jose@gmail.com> wrote: >> > >> > >> > >> > >> > >> > >> > >> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia >> domingo, 23/04/2023 à(s) 20:55: >> > >> >> >> > >> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer >> > >> >> <Christoph.Lauer@email.de> wrote: >> > >> >> > >> > >> >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield: >> > >> >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via >> > >> >> > > lists.openembedded.org >> > >> >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: >> > >> >> > >> >> > >> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie >> > >> >> > >> <richard.purdie@linuxfoundation.org> wrote: >> > >> >> > >>> >> > >> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: >> > >> >> > >>>> Hi, >> > >> >> > >>>> >> > >> >> > >>>> Not related with the previous discussion but just for >> > >> >> > >>>> your information. >> > >> >> > >>>> The rm_work.bbclass has an exception for the kernel >> recipes [1]. >> > >> >> > >>>> So I don't understand why we can't do the same for the >> make-mod- >> > >> >> > >>>> scripts >> > >> >> > >>>> who is the twin brother of all these kernel recipes. >> > >> >> > >>>> >> > >> >> > >>>> [1] >> > >> >> > >>>> >> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 >> > >> >> > >>> >> > >> >> > >>> Ideally we wouldn't be doing this for the kernel recipes. >> > >> >> > >>> >> > >> >> > >>> There is also a big difference to that and the proposed >> patch. The >> > >> >> > >>> proposed patch was preserving a specific directory rather >> than an >> > >> >> > >>> entire recipe. Removing the task stamps but leaving a small >> piece of >> > >> >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS >> for a >> > >> >> > >>> specific recipe. The former is not tested and will break >> things. The >> > >> >> > >>> latter is better tolerated by bitbake. >> > >> >> > >> >> > >> >> > >> Agreed. >> > >> >> > >> >> > >> >> > >> Plus, I am working on this now. >> > >> >> > >> >> > >> >> > >> I have static linking of the scripts/tools working, but what >> I haven't >> > >> >> > >> figured out is how to do that without patching the Makefiles. >> > >> >> > >> >> > >> >> > > >> > >> >> > > It turned out to be quite the battle to get older kernels >> what was >> > >> >> > > required for static linking of the tools. >> > >> >> > > >> > >> >> > > Attached is my WIP patch. I'm out of the office early next >> week, but >> > >> >> > > will revisit it once I'm back. >> > >> >> > > >> > >> >> > > Bruce >> > >> >> > > >> > >> >> > >> Next up will be some rpath trickery. >> > >> >> > >> >> > >> >> > >> Bruce >> > >> >> > >> >> > >> >> > >>> >> > >> >> > >>> So yes, we could do the same. I'm sure there will be other >> recipes >> > >> >> > >>> people want to preserve for other reasons. Where do we draw >> the line? >> > >> >> > >>> We could preserve everything and drop rm_work, then we >> wouldn't have >> > >> >> > >>> these problems? :) >> > >> >> > >>> >> > >> >> > >>> Cheers, >> > >> >> > >>> >> > >> >> > >>> Richard >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> -- >> > >> >> > >> - Thou shalt not follow the NULL pointer, for chaos and >> madness await >> > >> >> > >> thee at its end >> > >> >> > >> - "Use the force Harry" - Gandalf, Star Trek II >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > > >> > >> >> > > >> > >> >> > >> > >> >> > Thank you for your work, I see you put some time and effort >> into it. >> > >> >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel >> version 5.19 >> > >> >> >> > >> >> Yes, I realize that and documented it in the patch ... but I also >> > >> >> tested on pre-5.19 kernels and what I have in the patch works. >> Did it >> > >> >> not work in your testing ? >> > >> > >> > >> > >> > >> > I will test the patch on a couple of kernel versions with some of >> them pre-5.19 >> > >> > but all in 5 major versions. >> > >> > I will say something about my results later this week. >> > >> >> > >> 5.15-stable also has the pkg-config changes >> > >> >> > >> Bruce >> > >> >> > >> > >> > >> > Thanks for working on this one. >> > >> > >> > >> > Jose >> > >> > >> > >> >> >> > >> >> >> > >> >> > (see kernel patch [1]), so we need a way to call 'pkg-config >> --static' >> > >> >> > with pre-5.19 kernels. A way without modifying the Makefile >> would be to >> > >> >> > modify openssls pkg-config in recipe-sysroot-native of >> make-mod-script, >> > >> >> > so 'pkg-config --libs' actually shows the dependencies of >> 'pkg-config >> > >> >> > --static --libs', but it's a bit hacky. >> > >> >> >> > >> >> Already considered, and discarded. That's not going to fly. >> > >> >> >> > >> >> > >> > >> >> > Also fully-static executables still need the same glibc during >> runtime >> > >> >> > that they were built with, which makes them error-prone and is >> generally >> > >> >> > discouraged. As an alternative, we could build dynamic >> executables that >> > >> >> > use the static libcrypto library. The linker links by default >> against >> > >> >> > the shared library, so we could remove them from >> recipe-sysroot-native >> > >> >> > to force linking against the static library (again, somewhat >> hacky). >> > >> >> >> > >> >> Also considered and discarded. >> > >> >> >> > >> >> As do the dynamically linked ones for the c runtime. We aren't >> talking >> > >> >> about using these outside of a single build and they are >> generated on >> > >> >> the fly, so again, there's very little concern about runtimes >> changing >> > >> >> after linking.. There's less risk in static than in the >> alternatives. >> > >> >> >> > >> >> Bruce >> > >> >> >> > >> >> >> > >> >> > >> > >> >> > [1] >> > >> >> > >> https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 >> > >> >> >> > >> >> >> > >> >> >> > >> >> -- >> > >> >> - Thou shalt not follow the NULL pointer, for chaos and madness >> await >> > >> >> thee at its end >> > >> >> - "Use the force Harry" - Gandalf, Star Trek II >> > >> > >> > >> > >> > >> > >> > >> > -- >> > >> > Best regards, >> > >> > >> > >> > José Quaresma >> > >> >> > >> >> > >> >> > >> -- >> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> > >> thee at its end >> > >> - "Use the force Harry" - Gandalf, Star Trek II >> > > >> > > >> > > >> > > -- >> > > Best regards, >> > > >> > > José Quaresma >> > >> > >> > >> > -- >> > - Thou shalt not follow the NULL pointer, for chaos and madness await >> > thee at its end >> > - "Use the force Harry" - Gandalf, Star Trek II >> > >> > >> > >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II >> > > > -- > Best regards, > > José Quaresma > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#180805): > https://lists.openembedded.org/g/openembedded-core/message/180805 > Mute This Topic: https://lists.openembedded.org/mt/98296212/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Fri, May 5, 2023 at 6:24 AM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > Hi Bruce, > > Jose Quaresma via lists.openembedded.org <quaresma.jose=gmail.com@lists.openembedded.org> escreveu no dia quarta, 3/05/2023 à(s) 11:09: >> >> >> >> Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, 2/05/2023 à(s) 22:12: >>> >>> Attached is v2 of the patch. I've consolidated the suggested changes. >>> >>> I'm soaking it a bit longer, and then will send it as part of my next >>> consolidated pull request. >> >> >> I will do some more tests with the v2 and post my comment later if anything new comes up. > > > Nothing new and the v2 patch works pretty well in my kernels. > Awesome. Thanks for the test and update! Bruce > Jose > >> >> >> Jose >> >>> >>> >>> Bruce >>> >>> On Thu, Apr 27, 2023 at 9:26 PM Bruce Ashfield via >>> lists.openembedded.org >>> <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: >>> > >>> > On Thu, Apr 27, 2023 at 6:32 PM Jose Quaresma <quaresma.jose@gmail.com> wrote: >>> > > >>> > > Hi Bruce, >>> > > >>> > > I have been testing your patch and have some comments. >>> > > In some of my kernels I don't have the pkg-config changes and so I have some fails linking the scripts/sign-file >>> > > because for static linking with the libcrypto we need the -ldl -pthread. >>> > > >>> > > To fix my build I need to override the CRYPTO_LIBS in my kernel because they use the hardcoded pkg-config >>> > > where it is not possible to pass the --static argument. >>> > > >>> > > With following change on top of your patch I can build moist of my kernels: >>> > > >>> > > export HOSTLDFLAGS="-lz" >>> > > >>> > > + HOSTPKG_CONFIG="pkg-config --static" >>> > > + # override CRYPTO_LIBS since HOSTPKG_CONFIG lands only in v5.19-rc1 >>> > > + # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 >>> > > + CRYPTO_LIBS="$(pkg-config --static --libs libcrypto 2>/dev/null || echo -lcrypto)" >>> > > + >>> > > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS >>> > > for t in prepare scripts_basic scripts; do >>> > > oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \ >>> > > AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \ >>> > > - HOSTPKG_CONFIG="pkg-config --static" \ >>> > > + HOSTPKG_CONFIG="${HOSTPKG_CONFIG}" CRYPTO_LIBS="${CRYPTO_LIBS}" \ >>> > > -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t >>> > > done >>> > > >>> > > >>> > > I think belive that the LIBELF_LIBS needs the same fix for the cases where HOSTPKG_CONFIG is not available. >>> > > Also I think there is a typo in the LIBELF_LIBS because you first populate it with pkg-config but on the export the variable is redefined >>> > > with the HOST_LIBELF_LIBS. I made a litle change too on that: >>> > > >>> > > # for pre-5.15 kernels >>> > > - LIBELF_LIBS=$(pkg-config libelf --libs 2>/dev/null || echo -lelf) >>> > > - export LIBELF_LIBS="$HOST_LIBELF_LIBS -lz" >>> > > + LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null || echo -lelf) >>> > > + export LIBELF_LIBS="$LIBELF_LIBS -lz" >>> > > export HOSTLDFLAGS="-lz" >>> > > >>> > > Thanks for you help. >>> > >>> > Those are definitely plausible tweaks to the patch, I was providing >>> > the two techniques for that reason, and you've used them >>> > appropriately. >>> > >>> > Let me roll your changes into my patch, re-test and I'll submit it to >>> > the mailing list as a v2. >>> > >>> > Thanks for the testing, and fixup, I knew there would be things missing! :) >>> > >>> > Bruce >>> > >>> > > >>> > > Jose >>> > > >>> > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia segunda, 24/04/2023 à(s) 20:25: >>> > >> >>> > >> On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma <quaresma.jose@gmail.com> wrote: >>> > >> > >>> > >> > >>> > >> > >>> > >> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo, 23/04/2023 à(s) 20:55: >>> > >> >> >>> > >> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer >>> > >> >> <Christoph.Lauer@email.de> wrote: >>> > >> >> > >>> > >> >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield: >>> > >> >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via >>> > >> >> > > lists.openembedded.org >>> > >> >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: >>> > >> >> > >> >>> > >> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie >>> > >> >> > >> <richard.purdie@linuxfoundation.org> wrote: >>> > >> >> > >>> >>> > >> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote: >>> > >> >> > >>>> Hi, >>> > >> >> > >>>> >>> > >> >> > >>>> Not related with the previous discussion but just for >>> > >> >> > >>>> your information. >>> > >> >> > >>>> The rm_work.bbclass has an exception for the kernel recipes [1]. >>> > >> >> > >>>> So I don't understand why we can't do the same for the make-mod- >>> > >> >> > >>>> scripts >>> > >> >> > >>>> who is the twin brother of all these kernel recipes. >>> > >> >> > >>>> >>> > >> >> > >>>> [1] >>> > >> >> > >>>> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168 >>> > >> >> > >>> >>> > >> >> > >>> Ideally we wouldn't be doing this for the kernel recipes. >>> > >> >> > >>> >>> > >> >> > >>> There is also a big difference to that and the proposed patch. The >>> > >> >> > >>> proposed patch was preserving a specific directory rather than an >>> > >> >> > >>> entire recipe. Removing the task stamps but leaving a small piece of >>> > >> >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a >>> > >> >> > >>> specific recipe. The former is not tested and will break things. The >>> > >> >> > >>> latter is better tolerated by bitbake. >>> > >> >> > >> >>> > >> >> > >> Agreed. >>> > >> >> > >> >>> > >> >> > >> Plus, I am working on this now. >>> > >> >> > >> >>> > >> >> > >> I have static linking of the scripts/tools working, but what I haven't >>> > >> >> > >> figured out is how to do that without patching the Makefiles. >>> > >> >> > >> >>> > >> >> > > >>> > >> >> > > It turned out to be quite the battle to get older kernels what was >>> > >> >> > > required for static linking of the tools. >>> > >> >> > > >>> > >> >> > > Attached is my WIP patch. I'm out of the office early next week, but >>> > >> >> > > will revisit it once I'm back. >>> > >> >> > > >>> > >> >> > > Bruce >>> > >> >> > > >>> > >> >> > >> Next up will be some rpath trickery. >>> > >> >> > >> >>> > >> >> > >> Bruce >>> > >> >> > >> >>> > >> >> > >>> >>> > >> >> > >>> So yes, we could do the same. I'm sure there will be other recipes >>> > >> >> > >>> people want to preserve for other reasons. Where do we draw the line? >>> > >> >> > >>> We could preserve everything and drop rm_work, then we wouldn't have >>> > >> >> > >>> these problems? :) >>> > >> >> > >>> >>> > >> >> > >>> Cheers, >>> > >> >> > >>> >>> > >> >> > >>> Richard >>> > >> >> > >> >>> > >> >> > >> >>> > >> >> > >> >>> > >> >> > >> -- >>> > >> >> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await >>> > >> >> > >> thee at its end >>> > >> >> > >> - "Use the force Harry" - Gandalf, Star Trek II >>> > >> >> > >> >>> > >> >> > >> >>> > >> >> > >> >>> > >> >> > > >>> > >> >> > > >>> > >> >> > >>> > >> >> > Thank you for your work, I see you put some time and effort into it. >>> > >> >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19 >>> > >> >> >>> > >> >> Yes, I realize that and documented it in the patch ... but I also >>> > >> >> tested on pre-5.19 kernels and what I have in the patch works. Did it >>> > >> >> not work in your testing ? >>> > >> > >>> > >> > >>> > >> > I will test the patch on a couple of kernel versions with some of them pre-5.19 >>> > >> > but all in 5 major versions. >>> > >> > I will say something about my results later this week. >>> > >> >>> > >> 5.15-stable also has the pkg-config changes >>> > >> >>> > >> Bruce >>> > >> >>> > >> > >>> > >> > Thanks for working on this one. >>> > >> > >>> > >> > Jose >>> > >> > >>> > >> >> >>> > >> >> >>> > >> >> > (see kernel patch [1]), so we need a way to call 'pkg-config --static' >>> > >> >> > with pre-5.19 kernels. A way without modifying the Makefile would be to >>> > >> >> > modify openssls pkg-config in recipe-sysroot-native of make-mod-script, >>> > >> >> > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config >>> > >> >> > --static --libs', but it's a bit hacky. >>> > >> >> >>> > >> >> Already considered, and discarded. That's not going to fly. >>> > >> >> >>> > >> >> > >>> > >> >> > Also fully-static executables still need the same glibc during runtime >>> > >> >> > that they were built with, which makes them error-prone and is generally >>> > >> >> > discouraged. As an alternative, we could build dynamic executables that >>> > >> >> > use the static libcrypto library. The linker links by default against >>> > >> >> > the shared library, so we could remove them from recipe-sysroot-native >>> > >> >> > to force linking against the static library (again, somewhat hacky). >>> > >> >> >>> > >> >> Also considered and discarded. >>> > >> >> >>> > >> >> As do the dynamically linked ones for the c runtime. We aren't talking >>> > >> >> about using these outside of a single build and they are generated on >>> > >> >> the fly, so again, there's very little concern about runtimes changing >>> > >> >> after linking.. There's less risk in static than in the alternatives. >>> > >> >> >>> > >> >> Bruce >>> > >> >> >>> > >> >> >>> > >> >> > >>> > >> >> > [1] >>> > >> >> > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5 >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> -- >>> > >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await >>> > >> >> thee at its end >>> > >> >> - "Use the force Harry" - Gandalf, Star Trek II >>> > >> > >>> > >> > >>> > >> > >>> > >> > -- >>> > >> > Best regards, >>> > >> > >>> > >> > José Quaresma >>> > >> >>> > >> >>> > >> >>> > >> -- >>> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await >>> > >> thee at its end >>> > >> - "Use the force Harry" - Gandalf, Star Trek II >>> > > >>> > > >>> > > >>> > > -- >>> > > Best regards, >>> > > >>> > > José Quaresma >>> > >>> > >>> > >>> > -- >>> > - Thou shalt not follow the NULL pointer, for chaos and madness await >>> > thee at its end >>> > - "Use the force Harry" - Gandalf, Star Trek II >>> > >>> > >>> > >>> >>> >>> -- >>> - Thou shalt not follow the NULL pointer, for chaos and madness await >>> thee at its end >>> - "Use the force Harry" - Gandalf, Star Trek II >> >> >> >> -- >> Best regards, >> >> José Quaresma >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#180805): https://lists.openembedded.org/g/openembedded-core/message/180805 >> Mute This Topic: https://lists.openembedded.org/mt/98296212/5052612 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [quaresma.jose@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> > > > -- > Best regards, > > José Quaresma
diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb index 28e0807d1d..0e24efc597 100644 --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb @@ -32,3 +32,6 @@ do_configure() { -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t done } + +# keep native libraries required for module signing +RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native"